!cp /content/drive/MyDrive/Colab/Data/'Готовые задачи'/Калининград/participants/train/train.csv ./
!cp /content/drive/MyDrive/Colab/Data/'Готовые задачи'/Калининград/participants/test/test.csv ./
cp: cannot stat '/content/drive/MyDrive/Colab/Data/'\'''$'\320\223\320\276\321\202\320\276\320\262\321\213\320\265'' '$'\320\267\320\260\320\264\320\260\321\207\320\270'\''/'$'\320\232\320\260\320\273\320\270\320\275\320\270\320\275\320\263\321\200\320\260\320\264''/participants/train/train.csv': No such file or directory cp: cannot stat '/content/drive/MyDrive/Colab/Data/'\'''$'\320\223\320\276\321\202\320\276\320\262\321\213\320\265'' '$'\320\267\320\260\320\264\320\260\321\207\320\270'\''/'$'\320\232\320\260\320\273\320\270\320\275\320\270\320\275\320\263\321\200\320\260\320\264''/participants/test/test.csv': No such file or directory
#Установка catboost
!pip install catboost
Collecting catboost
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Downloading catboost-1.0.6-cp38-none-win_amd64.whl (73.9 MB) Collecting plotly Downloading plotly-5.9.0-py2.py3-none-any.whl (15.2 MB) Requirement already satisfied: scipy in d:\programdata\anaconda3\lib\site-packages (from catboost) (1.7.3) Requirement already satisfied: pandas>=0.24.0 in d:\programdata\anaconda3\lib\site-packages (from catboost) (1.4.1) Requirement already satisfied: matplotlib in d:\programdata\anaconda3\lib\site-packages (from catboost) (3.5.1) Requirement already satisfied: numpy>=1.16.0 in d:\programdata\anaconda3\lib\site-packages (from catboost) (1.20.3) Requirement already satisfied: six in d:\programdata\anaconda3\lib\site-packages (from catboost) (1.16.0) Collecting graphviz Downloading graphviz-0.20.1-py3-none-any.whl (47 kB) Requirement already satisfied: pytz>=2020.1 in d:\programdata\anaconda3\lib\site-packages (from pandas>=0.24.0->catboost) (2021.3) Requirement already satisfied: python-dateutil>=2.8.1 in d:\programdata\anaconda3\lib\site-packages (from pandas>=0.24.0->catboost) (2.8.2) Requirement already satisfied: fonttools>=4.22.0 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->catboost) (4.25.0) Requirement already satisfied: packaging>=20.0 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->catboost) (21.3) Requirement already satisfied: pillow>=6.2.0 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->catboost) (8.4.0) Requirement already satisfied: kiwisolver>=1.0.1 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->catboost) (1.3.2) Requirement already satisfied: pyparsing>=2.2.1 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->catboost) (3.0.4) Requirement already satisfied: cycler>=0.10 in d:\programdata\anaconda3\lib\site-packages (from matplotlib->catboost) (0.11.0) Collecting tenacity>=6.2.0 Using cached tenacity-8.0.1-py3-none-any.whl (24 kB) Installing collected packages: tenacity, plotly, graphviz, catboost Successfully installed catboost-1.0.6 graphviz-0.20.1 plotly-5.9.0 tenacity-8.0.1
#import необходимых модулей
import pandas as pd
from catboost import CatBoostRegressor, Pool
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
#Считывание данных в DataFrame
train = pd.read_csv('train.csv', sep=';', index_col=None, dtype={'PATIENT_SEX':str, 'MKB_CODE':str, 'ADRES':str, 'VISIT_MONTH_YEAR':str, 'AGE_CATEGORY':str, 'PATIENT_ID_COUNT':int})
test = pd.read_csv('test.csv', sep=';', index_col=None, dtype={'PATIENT_SEX':str, 'MKB_CODE':str, 'ADRES':str, 'VISIT_MONTH_YEAR':str, 'AGE_CATEGORY':str})
#Отделение меток от данных
X = train[['PATIENT_SEX', 'MKB_CODE', 'ADRES', 'VISIT_MONTH_YEAR', 'AGE_CATEGORY']]
y = train[['PATIENT_ID_COUNT']]
train['month_year'] = pd.to_datetime(train.loc[:, 'VISIT_MONTH_YEAR'], format='%m.%y', errors='ignore')
train
| PATIENT_SEX | MKB_CODE | ADRES | VISIT_MONTH_YEAR | AGE_CATEGORY | PATIENT_ID_COUNT | month_year | |
|---|---|---|---|---|---|---|---|
| 0 | 0 | A00.0 | Гурьевск | 08.21 | young | 1 | 2021-08-01 |
| 1 | 0 | A00.0 | Калининград | 03.20 | children | 1 | 2020-03-01 |
| 2 | 0 | A00 | Гусев | 03.19 | children | 1 | 2019-03-01 |
| 3 | 0 | A00 | Калининград | 01.22 | children | 1 | 2022-01-01 |
| 4 | 0 | A00 | Калининград | 02.18 | children | 1 | 2018-02-01 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 2212388 | 1 | Z99.1 | Гурьевск | 12.21 | children | 1 | 2021-12-01 |
| 2212389 | 1 | Z99.8 | Калининград | 10.21 | young | 1 | 2021-10-01 |
| 2212390 | 1 | Z99.9 | Калининград | 04.19 | children | 2 | 2019-04-01 |
| 2212391 | 1 | Z99.9 | Калининград | 08.19 | children | 1 | 2019-08-01 |
| 2212392 | 1 | Z99.9 | Калининград | 11.19 | children | 1 | 2019-11-01 |
2212393 rows × 7 columns
unique_months = X['VISIT_MONTH_YEAR'].unique().astype('str')
unique_months
array(['08.21', '03.20', '03.19', '01.22', '02.18', '03.22', '07.18',
'09.21', '10.19', '12.21', '05.18', '01.19', '05.19', '12.20',
'06.18', '10.20', '06.20', '11.19', '12.19', '02.19', '02.22',
'04.19', '05.21', '07.21', '01.20', '01.18', '02.20', '03.18',
'03.21', '04.18', '04.20', '04.21', '05.20', '06.19', '06.21',
'07.19', '07.20', '08.18', '08.19', '08.20', '09.18', '09.19',
'09.20', '10.18', '10.21', '11.18', '11.20', '11.21', '12.18',
'01.21', '02.21'], dtype='<U5')
df_uniq_months = pd.to_datetime(unique_months, format='%m.%y', errors='ignore')
df_uniq_months.sort_values()
DatetimeIndex(['2018-01-01', '2018-02-01', '2018-03-01', '2018-04-01',
'2018-05-01', '2018-06-01', '2018-07-01', '2018-08-01',
'2018-09-01', '2018-10-01', '2018-11-01', '2018-12-01',
'2019-01-01', '2019-02-01', '2019-03-01', '2019-04-01',
'2019-05-01', '2019-06-01', '2019-07-01', '2019-08-01',
'2019-09-01', '2019-10-01', '2019-11-01', '2019-12-01',
'2020-01-01', '2020-02-01', '2020-03-01', '2020-04-01',
'2020-05-01', '2020-06-01', '2020-07-01', '2020-08-01',
'2020-09-01', '2020-10-01', '2020-11-01', '2020-12-01',
'2021-01-01', '2021-02-01', '2021-03-01', '2021-04-01',
'2021-05-01', '2021-06-01', '2021-07-01', '2021-08-01',
'2021-09-01', '2021-10-01', '2021-11-01', '2021-12-01',
'2022-01-01', '2022-02-01', '2022-03-01'],
dtype='datetime64[ns]', freq=None)
df_uniq_months.sort_values(ascending=True)
DatetimeIndex(['2018-01-01', '2018-02-01', '2018-03-01', '2018-04-01',
'2018-05-01', '2018-06-01', '2018-07-01', '2018-08-01',
'2018-09-01', '2018-10-01', '2018-11-01', '2018-12-01',
'2019-01-01', '2019-02-01', '2019-03-01', '2019-04-01',
'2019-05-01', '2019-06-01', '2019-07-01', '2019-08-01',
'2019-09-01', '2019-10-01', '2019-11-01', '2019-12-01',
'2020-01-01', '2020-02-01', '2020-03-01', '2020-04-01',
'2020-05-01', '2020-06-01', '2020-07-01', '2020-08-01',
'2020-09-01', '2020-10-01', '2020-11-01', '2020-12-01',
'2021-01-01', '2021-02-01', '2021-03-01', '2021-04-01',
'2021-05-01', '2021-06-01', '2021-07-01', '2021-08-01',
'2021-09-01', '2021-10-01', '2021-11-01', '2021-12-01',
'2022-01-01', '2022-02-01', '2022-03-01'],
dtype='datetime64[ns]', freq=None)
uniq_city = X['ADRES'].unique()
print(uniq_city)
['Гурьевск' 'Калининград' 'Гусев' 'Нестеров' 'Партизанское' 'Садовое' 'Славск' 'Ясная Поляна' 'Гвардейск' 'Холмогоровка' 'Багратионовск' 'Большое Исаково' 'Зеленоградск' 'Калининградская' 'Колосовка' 'Краснознаменск' 'Невское' 'Неман' 'Пионерский' 'Правдинск' 'Прибрежный' 'СТ Железнодорожник' 'СТ Искра ул. Тюльпановая' 'СТ Победа' 'Светлый' 'Советск' 'Совхозное' 'Черняховск' 'Балтийск' 'Малое Васильково' 'Полесск' 'Сосновка' 'Ладушкин' 'Малинники' 'Раздольное' 'Озерск' 'Волочаевское' 'Мамоново' 'Нивенское' 'Светлогорск' 'Васильково' 'Голубево' 'Долгоруково' 'Донское' 'Дружный' 'Заозерье' 'Корнево' 'Ласкино' 'Луговое' 'Люблино' 'Малиновка' 'Малое Исаково' 'Матросово' 'Озерки' 'Переславское' 'Пятидорожное' 'СТ Колосок' 'СТ Радуга' 'Саранское' 'Славинск' 'Славянское' 'Ушаково' 'Янтарный' 'Ясное' 'Илюшино' 'Шоссейное' 'Южный' 'Петрово' 'Некрасово' 'Березовка' 'Большаково' 'Высокое' 'Домново' 'Дорожный' 'Железнодорожный' 'Знаменск' 'Коврово' 'Комсомольск' 'Константиновка' 'Крылово' 'Кумачево' 'Майское' 'Мельниково' 'Муромское' 'Низовье' 'Новодорожный' 'Новостроево' 'Первомайское' 'Пригородное' 'Приморск' 'Родники' 'Романово' 'Рощино' 'Рыбачий' 'СТ Чайка' 'Тимирязево' 'Храброво' 'Лесное' 'Взморье' 'Приморье' 'Рассвет' 'Залесье' 'Гурьевский' 'Фурманово' 'Заливино' 'Малое Луговое' 'Покровское' 'Прибрежное' 'Рыбное' 'Орловка' 'Владимирово' 'Добрино' 'Заостровье' 'Кострово' 'Маршальское' 'Ново-Московское' 'Северный' 'Тургенево']
# TODO: get population by city name
# TODO: get lat, lon by city name
# TODO: get weather record by city name
# TODO: research periodicity
# TODO: EDA
train.head()
| PATIENT_SEX | MKB_CODE | ADRES | VISIT_MONTH_YEAR | AGE_CATEGORY | PATIENT_ID_COUNT | |
|---|---|---|---|---|---|---|
| 0 | 0 | A00.0 | Гурьевск | 08.21 | young | 1 |
| 1 | 0 | A00.0 | Калининград | 03.20 | children | 1 |
| 2 | 0 | A00 | Гусев | 03.19 | children | 1 |
| 3 | 0 | A00 | Калининград | 01.22 | children | 1 |
| 4 | 0 | A00 | Калининград | 02.18 | children | 1 |
train.groupby('PATIENT_SEX').sum('PATIENT_ID_COUNT')
| PATIENT_ID_COUNT | |
|---|---|
| PATIENT_SEX | |
| 0 | 7988811 |
| 1 | 4948939 |
train.groupby('AGE_CATEGORY').sum('PATIENT_ID_COUNT').sort_values(by='PATIENT_ID_COUNT', ascending=False)
| PATIENT_ID_COUNT | |
|---|---|
| AGE_CATEGORY | |
| children | 4187970 |
| young | 3100989 |
| elderly | 2670328 |
| middleage | 1942558 |
| old | 888505 |
| centenarians | 147400 |
train.groupby('ADRES').sum('PATIENT_ID_COUNT').sort_values(by='PATIENT_ID_COUNT', ascending=False)
| PATIENT_ID_COUNT | |
|---|---|
| ADRES | |
| Калининград | 8615298 |
| Гурьевск | 559127 |
| Пионерский | 306848 |
| Советск | 291640 |
| Черняховск | 263656 |
| ... | ... |
| Ясная Поляна | 3432 |
| Ново-Московское | 3409 |
| Рыбачий | 3405 |
| Гурьевский | 3337 |
| СТ Железнодорожник | 3176 |
118 rows × 1 columns
train.groupby('VISIT_MONTH_YEAR').sum('PATIENT_ID_COUNT').sort_values(by='PATIENT_ID_COUNT', ascending=False)
| PATIENT_ID_COUNT | |
|---|---|
| VISIT_MONTH_YEAR | |
| 10.21 | 347865 |
| 12.21 | 345437 |
| 11.21 | 328531 |
| 09.21 | 328436 |
| 07.21 | 316501 |
| 08.21 | 313895 |
| 04.21 | 313055 |
| 06.21 | 311045 |
| 10.19 | 304796 |
| 05.21 | 294018 |
| 04.19 | 290365 |
| 10.18 | 287364 |
| 12.19 | 282251 |
| 09.19 | 280559 |
| 02.19 | 279467 |
| 03.21 | 275785 |
| 03.19 | 275218 |
| 11.18 | 272947 |
| 11.19 | 269928 |
| 10.20 | 263960 |
| 03.18 | 263285 |
| 07.19 | 262111 |
| 03.22 | 261884 |
| 09.20 | 261412 |
| 12.18 | 259683 |
| 08.19 | 258120 |
| 04.18 | 258038 |
| 02.18 | 255036 |
| 09.18 | 253018 |
| 05.19 | 243530 |
| 08.18 | 242724 |
| 02.20 | 241591 |
| 06.19 | 240687 |
| 01.19 | 240650 |
| 07.18 | 239620 |
| 06.18 | 238109 |
| 05.18 | 237447 |
| 12.20 | 234672 |
| 03.20 | 233576 |
| 08.20 | 225479 |
| 11.20 | 224981 |
| 02.21 | 223190 |
| 01.18 | 221854 |
| 07.20 | 217798 |
| 02.22 | 196348 |
| 01.22 | 190701 |
| 01.20 | 189966 |
| 06.20 | 173373 |
| 01.21 | 150000 |
| 04.20 | 111634 |
| 05.20 | 105810 |
import matplotlib.pyplot as plt
mkb_count = train.groupby('MKB_CODE').sum('PATIENT_ID_COUNT').reset_index()
mkb_count.describe()
| PATIENT_ID_COUNT | |
|---|---|
| count | 7644.000000 |
| mean | 1692.536630 |
| std | 17989.725347 |
| min | 1.000000 |
| 25% | 6.000000 |
| 50% | 40.000000 |
| 75% | 284.000000 |
| max | 919789.000000 |
mkb_count.index
RangeIndex(start=0, stop=7644, step=1)
mkb_count.sort_values(by=['PATIENT_ID_COUNT'], ascending=False, inplace=True)
mkb_count.head()
| MKB_CODE | PATIENT_ID_COUNT | |
|---|---|---|
| 2847 | J06.9 | 919789 |
| 7268 | Z25.8 | 663351 |
| 7123 | Z00.0 | 611981 |
| 2506 | I11.9 | 350152 |
| 3066 | K02.1 | 316946 |
mkb_count.query('PATIENT_ID_COUNT > 10000')
| MKB_CODE | PATIENT_ID_COUNT | |
|---|---|---|
| 2847 | J06.9 | 919789 |
| 7268 | Z25.8 | 663351 |
| 7123 | Z00.0 | 611981 |
| 2506 | I11.9 | 350152 |
| 3066 | K02.1 | 316946 |
| ... | ... | ... |
| 2385 | H65.1 | 10087 |
| 3256 | K42.9 | 10079 |
| 2118 | G93.8 | 10054 |
| 1575 | F11.2 | 10050 |
| 4036 | M54.9 | 10019 |
202 rows × 2 columns
mkb_count.query('PATIENT_ID_COUNT > 10000').iloc[0:10,0]
2847 J06.9 7268 Z25.8 7123 Z00.0 2506 I11.9 3066 K02.1 7200 Z11.5 7150 Z02.7 7124 Z00.1 2833 J02.9 7134 Z01.2 Name: MKB_CODE, dtype: object
type(mkb_count.query('PATIENT_ID_COUNT > 10000').iloc[:, 0])
pandas.core.series.Series
mkb_top_list = mkb_count.query('PATIENT_ID_COUNT > 10000').iloc[:, 0].tolist()
mkb_top_list
['J06.9', 'Z25.8', 'Z00.0', 'I11.9', 'K02.1', 'Z11.5', 'Z02.7', 'Z00.1', 'J02.9', 'Z01.2', 'K04.5', 'J00', 'M42.1', 'K04.0', 'Z01.4', 'J06.8', 'Z01.7', 'I67.8', 'Z00', 'J04.1', 'Z11.3', 'N60.1', 'Z01.0', 'K04.4', 'Z20.8', 'H52.1', 'K02.8', 'Z11.1', 'J20.9', 'Z01.1', 'Z02.5', 'N95.2', 'Z00.8', 'I67.9', 'I67.2', 'M54.4', 'Z71.2', 'Z25.1', 'O99.8', 'G93.4', 'H40.1', 'G90.8', 'Z01.8', 'K59.9', 'K07.3', 'I25.8', 'N40', 'H25.0', 'F10.2', 'H61.2', 'G90.9', 'Z02.0', 'I25.1', 'M54.5', 'Z76.0', 'K02.0', 'Z27.3', 'J18.9', 'I83.9', 'N76.0', 'K07.2', 'Z27.8', 'M95.8', 'M54.2', 'I20.8', 'I25.2', 'M42.9', 'M17.0', 'K29.9', 'C61', 'H90.3', 'Z34.8', 'E06.3', 'L23.9', 'J45.8', 'Z03.8', 'B01.9', 'H35.3', 'F20.0', 'E04.2', 'M54.1', 'C50.4', 'M17.1', 'H25.8', 'J35.0', 'K81.1', 'M21.4', 'I70.2', 'H52.0', 'H52.2', 'I10', 'B07', 'J03.9', 'O99.0', 'S93.4', 'D25.1', 'Z00.4', 'Z03.0', 'K29.5', 'M51.1', 'S60.0', 'N95.1', 'J04.2', 'I11.0', 'Z27.4', 'N72', 'M15.0', 'Z04.8', 'M21.0', 'Z20.1', 'K86.1', 'O26.0', 'J02', 'N20.0', 'Z24.6', 'Z34.0', 'S20.2', 'I69.3', 'D50.9', 'Z32.1', 'S52.5', 'N60.9', 'K00.7', 'Z51.8', 'K58.9', 'E03.8', 'J12.8', 'L70.0', 'K29.7', 'K05.3', 'G25.9', 'H65.0', 'S00.8', 'K82.8', 'H10.0', 'N76.1', 'E66.0', 'N81.2', 'D24', 'C50.9', 'J30.4', 'J30.0', 'T15.0', 'M54.6', 'S80.0', 'K21.0', 'N86', 'S90.0', 'J01.0', 'J31.2', 'E04.1', 'L24.9', 'B35.1', 'Z24.0', 'N92.6', 'K29.6', 'N41.1', 'H35.0', 'K59.0', 'B18.2', 'M15.9', 'Z27.1', 'J35.2', 'J34.2', 'N84.0', 'M19.0', 'R76.1', 'O23.5', 'M13.9', 'M77.3', 'I11', 'N60.8', 'I25.9', 'M41.1', 'M16.1', 'H52.4', 'D23.5', 'S61.0', 'N81.1', 'M16.0', 'K80.1', 'I67.4', 'H52.5', 'G35', 'J44.8', 'L40.0', 'N30.0', 'O20.0', 'K10.2', 'J06.0', 'J45.0', 'J42', 'Z03.3', 'N70.1', 'J04.0', 'J41.0', 'G20', 'H65.1', 'K42.9', 'G93.8', 'F11.2', 'M54.9']
J06.9 ОРВИ Z25.8 Необходимость иммунизации против одной из других вирусных болезней (коронавирус?) Z00.0 Общий медицинский осмотр I11.9
!pip install icd10-cm
Collecting icd10-cm
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Downloading icd10_cm-0.0.4-py2.py3-none-any.whl (675 kB) Installing collected packages: icd10-cm Successfully installed icd10-cm-0.0.4
import icd10
code = icd10.find("J20.0")
print(code.description) # Acute bronchitis due to Mycoplasma pneumoniae
if code.billable:
print(code, "is billable") # J20.0 is billable
print(code.chapter) # X
print(code.block) # J00-J99
print(code.block_description) # Diseases of the respiratory system
Acute bronchitis due to Mycoplasma pneumoniae J20.0 is billable X J00-J99 Diseases of the respiratory system
import icd10
if icd10.exists("J20.0"):
print("Exists")
Exists
mkb_top_name_dict = {}
for mkb in mkb_top_list:
code = icd10.find(mkb)
mkb_top_name_dict[mkb] = code.description if code is not None else 'NOT FOUND'
mkb_top_name_dict
{'J06.9': 'Acute upper respiratory infection, unspecified',
'Z25.8': 'NOT FOUND',
'Z00.0': 'Encounter for general adult medical examination',
'I11.9': 'Hypertensive heart disease without heart failure',
'K02.1': 'NOT FOUND',
'Z11.5': 'Encounter for screening for other viral diseases',
'Z02.7': 'Encounter for issue of medical certificate',
'Z00.1': 'Encounter for newborn, infant and child health examinations',
'J02.9': 'Acute pharyngitis, unspecified',
'Z01.2': 'Encounter for dental examination and cleaning',
'K04.5': 'Chronic apical periodontitis',
'J00': 'Acute nasopharyngitis [common cold]',
'M42.1': 'Adult osteochondrosis of spine',
'K04.0': 'Pulpitis',
'Z01.4': 'Encounter for gynecological examination',
'J06.8': 'NOT FOUND',
'Z01.7': 'NOT FOUND',
'I67.8': 'Other specified cerebrovascular diseases',
'Z00': 'Encounter for general examination without complaint, suspected or reported diagnosis',
'J04.1': 'Acute tracheitis',
'Z11.3': 'Encounter for screening for infections with a predominantly sexual mode of transmission',
'N60.1': 'Diffuse cystic mastopathy',
'Z01.0': 'Encounter for examination of eyes and vision',
'K04.4': 'Acute apical periodontitis of pulpal origin',
'Z20.8': 'Contact with and (suspected) exposure to other communicable diseases',
'H52.1': 'Myopia',
'K02.8': 'NOT FOUND',
'Z11.1': 'Encounter for screening for respiratory tuberculosis',
'J20.9': 'Acute bronchitis, unspecified',
'Z01.1': 'Encounter for examination of ears and hearing',
'Z02.5': 'Encounter for examination for participation in sport',
'N95.2': 'Postmenopausal atrophic vaginitis',
'Z00.8': 'Encounter for other general examination',
'I67.9': 'Cerebrovascular disease, unspecified',
'I67.2': 'Cerebral atherosclerosis',
'M54.4': 'Lumbago with sciatica',
'Z71.2': 'Person consulting for explanation of examination or test findings',
'Z25.1': 'NOT FOUND',
'O99.8': 'Other specified diseases and conditions complicating pregnancy, childbirth and the puerperium',
'G93.4': 'Other and unspecified encephalopathy',
'H40.1': 'Open-angle glaucoma',
'G90.8': 'Other disorders of autonomic nervous system',
'Z01.8': 'Encounter for other specified special examinations',
'K59.9': 'Functional intestinal disorder, unspecified',
'K07.3': 'NOT FOUND',
'I25.8': 'Other forms of chronic ischemic heart disease',
'N40': 'Benign prostatic hyperplasia',
'H25.0': 'Age-related incipient cataract',
'F10.2': 'Alcohol dependence',
'H61.2': 'Impacted cerumen',
'G90.9': 'Disorder of the autonomic nervous system, unspecified',
'Z02.0': 'Encounter for examination for admission to educational institution',
'I25.1': 'Atherosclerotic heart disease of native coronary artery',
'M54.5': 'Low back pain',
'Z76.0': 'Encounter for issue of repeat prescription',
'K02.0': 'NOT FOUND',
'Z27.3': 'NOT FOUND',
'J18.9': 'Pneumonia, unspecified organism',
'I83.9': 'Asymptomatic varicose veins of lower extremities',
'N76.0': 'Acute vaginitis',
'K07.2': 'NOT FOUND',
'Z27.8': 'NOT FOUND',
'M95.8': 'Other specified acquired deformities of musculoskeletal system',
'M54.2': 'Cervicalgia',
'I20.8': 'Other forms of angina pectoris',
'I25.2': 'Old myocardial infarction',
'M42.9': 'Spinal osteochondrosis, unspecified',
'M17.0': 'Bilateral primary osteoarthritis of knee',
'K29.9': 'Gastroduodenitis, unspecified',
'C61': 'Malignant neoplasm of prostate',
'H90.3': 'Sensorineural hearing loss, bilateral',
'Z34.8': 'Encounter for supervision of other normal pregnancy',
'E06.3': 'Autoimmune thyroiditis',
'L23.9': 'Allergic contact dermatitis, unspecified cause',
'J45.8': 'NOT FOUND',
'Z03.8': 'Encounter for observation for other suspected diseases and conditions ruled out',
'B01.9': 'Varicella without complication',
'H35.3': 'Degeneration of macula and posterior pole',
'F20.0': 'Paranoid schizophrenia',
'E04.2': 'Nontoxic multinodular goiter',
'M54.1': 'Radiculopathy',
'C50.4': 'Malignant neoplasm of upper-outer quadrant of breast',
'M17.1': 'Unilateral primary osteoarthritis of knee',
'H25.8': 'Other age-related cataract',
'J35.0': 'Chronic tonsillitis and adenoiditis',
'K81.1': 'Chronic cholecystitis',
'M21.4': 'Flat foot [pes planus] (acquired)',
'I70.2': 'Atherosclerosis of native arteries of the extremities',
'H52.0': 'Hypermetropia',
'H52.2': 'Astigmatism',
'I10': 'Essential (primary) hypertension',
'B07': 'Viral warts',
'J03.9': 'Acute tonsillitis, unspecified',
'O99.0': 'Anemia complicating pregnancy, childbirth and the puerperium',
'S93.4': 'Sprain of ankle',
'D25.1': 'Intramural leiomyoma of uterus',
'Z00.4': 'NOT FOUND',
'Z03.0': 'NOT FOUND',
'K29.5': 'Unspecified chronic gastritis',
'M51.1': 'Thoracic, thoracolumbar and lumbosacral intervertebral disc disorders with radiculopathy',
'S60.0': 'Contusion of finger without damage to nail',
'N95.1': 'Menopausal and female climacteric states',
'J04.2': 'Acute laryngotracheitis',
'I11.0': 'Hypertensive heart disease with heart failure',
'Z27.4': 'NOT FOUND',
'N72': 'Inflammatory disease of cervix uteri',
'M15.0': 'Primary generalized (osteo)arthritis',
'Z04.8': 'Encounter for examination and observation for other specified reasons',
'M21.0': 'Valgus deformity, not elsewhere classified',
'Z20.1': 'Contact with and (suspected) exposure to tuberculosis',
'K86.1': 'Other chronic pancreatitis',
'O26.0': 'Excessive weight gain in pregnancy',
'J02': 'Acute pharyngitis',
'N20.0': 'Calculus of kidney',
'Z24.6': 'NOT FOUND',
'Z34.0': 'Encounter for supervision of normal first pregnancy',
'S20.2': 'Contusion of thorax',
'I69.3': 'Sequelae of cerebral infarction',
'D50.9': 'Iron deficiency anemia, unspecified',
'Z32.1': 'NOT FOUND',
'S52.5': 'Fracture of lower end of radius',
'N60.9': 'Unspecified benign mammary dysplasia',
'K00.7': 'Teething syndrome',
'Z51.8': 'Encounter for other specified aftercare',
'K58.9': 'Irritable bowel syndrome without diarrhea',
'E03.8': 'Other specified hypothyroidism',
'J12.8': 'Other viral pneumonia',
'L70.0': 'Acne vulgaris',
'K29.7': 'Gastritis, unspecified',
'K05.3': 'Chronic periodontitis',
'G25.9': 'Extrapyramidal and movement disorder, unspecified',
'H65.0': 'Acute serous otitis media',
'S00.8': 'Superficial injury of other parts of head',
'K82.8': 'Other specified diseases of gallbladder',
'H10.0': 'Mucopurulent conjunctivitis',
'N76.1': 'Subacute and chronic vaginitis',
'E66.0': 'Obesity due to excess calories',
'N81.2': 'Incomplete uterovaginal prolapse',
'D24': 'Benign neoplasm of breast',
'C50.9': 'Malignant neoplasm of breast of unspecified site',
'J30.4': 'NOT FOUND',
'J30.0': 'Vasomotor rhinitis',
'T15.0': 'Foreign body in cornea',
'M54.6': 'Pain in thoracic spine',
'S80.0': 'Contusion of knee',
'K21.0': 'Gastro-esophageal reflux disease with esophagitis',
'N86': 'Erosion and ectropion of cervix uteri',
'S90.0': 'Contusion of ankle',
'J01.0': 'Acute maxillary sinusitis',
'J31.2': 'Chronic pharyngitis',
'E04.1': 'Nontoxic single thyroid nodule',
'L24.9': 'Irritant contact dermatitis, unspecified cause',
'B35.1': 'Tinea unguium',
'Z24.0': 'NOT FOUND',
'N92.6': 'Irregular menstruation, unspecified',
'K29.6': 'Other gastritis',
'N41.1': 'Chronic prostatitis',
'H35.0': 'Background retinopathy and retinal vascular changes',
'K59.0': 'Constipation',
'B18.2': 'Chronic viral hepatitis C',
'M15.9': 'Polyosteoarthritis, unspecified',
'Z27.1': 'NOT FOUND',
'J35.2': 'Hypertrophy of adenoids',
'J34.2': 'Deviated nasal septum',
'N84.0': 'Polyp of corpus uteri',
'M19.0': 'Primary osteoarthritis of other joints',
'R76.1': 'Nonspecific reaction to test for tuberculosis',
'O23.5': 'Infections of the genital tract in pregnancy',
'M13.9': 'NOT FOUND',
'M77.3': 'Calcaneal spur',
'I11': 'Hypertensive heart disease',
'N60.8': 'Other benign mammary dysplasias',
'I25.9': 'Chronic ischemic heart disease, unspecified',
'M41.1': 'Juvenile and adolescent idiopathic scoliosis',
'M16.1': 'Unilateral primary osteoarthritis of hip',
'H52.4': 'Presbyopia',
'D23.5': 'Other benign neoplasm of skin of trunk',
'S61.0': 'Open wound of thumb without damage to nail',
'N81.1': 'Cystocele',
'M16.0': 'Bilateral primary osteoarthritis of hip',
'K80.1': 'Calculus of gallbladder with other cholecystitis',
'I67.4': 'Hypertensive encephalopathy',
'H52.5': 'Disorders of accommodation',
'G35': 'Multiple sclerosis',
'J44.8': 'NOT FOUND',
'L40.0': 'Psoriasis vulgaris',
'N30.0': 'Acute cystitis',
'O20.0': 'Threatened abortion',
'K10.2': 'NOT FOUND',
'J06.0': 'Acute laryngopharyngitis',
'J45.0': 'NOT FOUND',
'J42': 'Unspecified chronic bronchitis',
'Z03.3': 'NOT FOUND',
'N70.1': 'Chronic salpingitis and oophoritis',
'J04.0': 'Acute laryngitis',
'J41.0': 'Simple chronic bronchitis',
'G20': "Parkinson's disease",
'H65.1': 'Other acute nonsuppurative otitis media',
'K42.9': 'Umbilical hernia without obstruction or gangrene',
'G93.8': 'Other specified disorders of brain',
'F11.2': 'Opioid dependence',
'M54.9': 'Dorsalgia, unspecified'}
mkb_top_name_dict['K58.9']
'Irritable bowel syndrome without diarrhea'
!pip install simple-icd-10
Collecting simple-icd-10
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Downloading simple_icd_10-2.0.1-py3-none-any.whl (173 kB) Installing collected packages: simple-icd-10 Successfully installed simple-icd-10-2.0.1
import simple_icd_10 as icd
icd.get_description("XII")
'Diseases of the skin and subcutaneous tissue'
mkb_top_name_simple_dict = {}
cnt_err = 0
for mkb in mkb_top_list:
try:
mkb_desc = icd.get_description(mkb)
mkb_top_name_simple_dict[mkb] = mkb_desc
except ValueError as e:
mkb_top_name_simple_dict[mkb] = 'NOT FOUND'
print(mkb)
cnt_err += 1
print(cnt_err)
K58.9 1
mkb_top_name_simple_dict
{'J06.9': 'Acute upper respiratory infection, unspecified',
'Z25.8': 'Need for immunization against other specified single viral diseases',
'Z00.0': 'General medical examination',
'I11.9': 'Hypertensive heart disease without (congestive) heart failure',
'K02.1': 'Caries of dentine',
'Z11.5': 'Special screening examination for other viral diseases',
'Z02.7': 'Issue of medical certificate',
'Z00.1': 'Routine child health examination',
'J02.9': 'Acute pharyngitis, unspecified',
'Z01.2': 'Dental examination',
'K04.5': 'Chronic apical periodontitis',
'J00': 'Acute nasopharyngitis [common cold]',
'M42.1': 'Adult osteochondrosis of spine',
'K04.0': 'Pulpitis',
'Z01.4': 'Gynaecological examination (general)(routine)',
'J06.8': 'Other acute upper respiratory infections of multiple sites',
'Z01.7': 'Laboratory examination',
'I67.8': 'Other specified cerebrovascular diseases',
'Z00': 'General examination and investigation of persons without complaint and reported diagnosis',
'J04.1': 'Acute tracheitis',
'Z11.3': 'Special screening examination for infections with a predominantly sexual mode of transmission',
'N60.1': 'Diffuse cystic mastopathy',
'Z01.0': 'Examination of eyes and vision',
'K04.4': 'Acute apical periodontitis of pulpal origin',
'Z20.8': 'Contact with and exposure to other communicable diseases',
'H52.1': 'Myopia',
'K02.8': 'Other dental caries',
'Z11.1': 'Special screening examination for respiratory tuberculosis',
'J20.9': 'Acute bronchitis, unspecified',
'Z01.1': 'Examination of ears and hearing',
'Z02.5': 'Examination for participation in sport',
'N95.2': 'Postmenopausal atrophic vaginitis',
'Z00.8': 'Other general examinations',
'I67.9': 'Cerebrovascular disease, unspecified',
'I67.2': 'Cerebral atherosclerosis',
'M54.4': 'Lumbago with sciatica',
'Z71.2': 'Person consulting for explanation of investigation findings',
'Z25.1': 'Need for immunization against influenza',
'O99.8': 'Other specified diseases and conditions complicating pregnancy, childbirth and the puerperium',
'G93.4': 'Encephalopathy, unspecified',
'H40.1': 'Primary open-angle glaucoma',
'G90.8': 'Other disorders of autonomic nervous system',
'Z01.8': 'Other specified special examinations',
'K59.9': 'Functional intestinal disorder, unspecified',
'K07.3': 'Anomalies of tooth position',
'I25.8': 'Other forms of chronic ischaemic heart disease',
'N40': 'Hyperplasia of prostate',
'H25.0': 'Senile incipient cataract',
'F10.2': 'Mental and behavioural disorders due to use of alcohol : dependence syndrome',
'H61.2': 'Impacted cerumen',
'G90.9': 'Disorder of autonomic nervous system, unspecified',
'Z02.0': 'Examination for admission to educational institution',
'I25.1': 'Atherosclerotic heart disease',
'M54.5': 'Low back pain',
'Z76.0': 'Issue of repeat prescription',
'K02.0': 'Caries limited to enamel',
'Z27.3': 'Need for immunization against diphtheria-tetanus-pertussis with poliomyelitis [DTP + polio]',
'J18.9': 'Pneumonia, unspecified',
'I83.9': 'Varicose veins of lower extremities without ulcer or inflammation',
'N76.0': 'Acute vaginitis',
'K07.2': 'Anomalies of dental arch relationship',
'Z27.8': 'Need for immunization against other combinations of infectious diseases',
'M95.8': 'Other specified acquired deformities of musculoskeletal system',
'M54.2': 'Cervicalgia',
'I20.8': 'Other forms of angina pectoris',
'I25.2': 'Old myocardial infarction',
'M42.9': 'Spinal osteochondrosis, unspecified',
'M17.0': 'Primary gonarthrosis, bilateral',
'K29.9': 'Gastroduodenitis, unspecified',
'C61': 'Malignant neoplasm of prostate',
'H90.3': 'Sensorineural hearing loss, bilateral',
'Z34.8': 'Supervision of other normal pregnancy',
'E06.3': 'Autoimmune thyroiditis',
'L23.9': 'Allergic contact dermatitis, unspecified cause',
'J45.8': 'Mixed asthma',
'Z03.8': 'Observation for other suspected diseases and conditions',
'B01.9': 'Varicella without complication',
'H35.3': 'Degeneration of macula and posterior pole',
'F20.0': 'Paranoid schizophrenia',
'E04.2': 'Nontoxic multinodular goitre',
'M54.1': 'Radiculopathy',
'C50.4': 'Malignant neoplasm: Upper-outer quadrant of breast',
'M17.1': 'Other primary gonarthrosis',
'H25.8': 'Other senile cataract',
'J35.0': 'Chronic tonsillitis',
'K81.1': 'Chronic cholecystitis',
'M21.4': 'Flat foot [pes planus] (acquired)',
'I70.2': 'Atherosclerosis of arteries of extremities',
'H52.0': 'Hypermetropia',
'H52.2': 'Astigmatism',
'I10': 'Essential (primary) hypertension',
'B07': 'Viral warts',
'J03.9': 'Acute tonsillitis, unspecified',
'O99.0': 'Anaemia complicating pregnancy, childbirth and the puerperium',
'S93.4': 'Sprain and strain of ankle',
'D25.1': 'Intramural leiomyoma of uterus',
'Z00.4': 'General psychiatric examination, not elsewhere classified',
'Z03.0': 'Observation for suspected tuberculosis',
'K29.5': 'Chronic gastritis, unspecified',
'M51.1': 'Lumbar and other intervertebral disc disorders with radiculopathy',
'S60.0': 'Contusion of finger(s) without damage to nail',
'N95.1': 'Menopausal and female climacteric states',
'J04.2': 'Acute laryngotracheitis',
'I11.0': 'Hypertensive heart disease with (congestive) heart failure',
'Z27.4': 'Need for immunization against measles-mumps-rubella [MMR]',
'N72': 'Inflammatory disease of cervix uteri',
'M15.0': 'Primary generalized (osteo)arthrosis',
'Z04.8': 'Examination and observation for other specified reasons',
'M21.0': 'Valgus deformity, not elsewhere classified',
'Z20.1': 'Contact with and exposure to tuberculosis',
'K86.1': 'Other chronic pancreatitis',
'O26.0': 'Excessive weight gain in pregnancy',
'J02': 'Acute pharyngitis',
'N20.0': 'Calculus of kidney',
'Z24.6': 'Need for immunization against viral hepatitis',
'Z34.0': 'Supervision of normal first pregnancy',
'S20.2': 'Contusion of thorax',
'I69.3': 'Sequelae of cerebral infarction',
'D50.9': 'Iron deficiency anaemia, unspecified',
'Z32.1': 'Pregnancy confirmed',
'S52.5': 'Fracture of lower end of radius',
'N60.9': 'Benign mammary dysplasia, unspecified',
'K00.7': 'Teething syndrome',
'Z51.8': 'Other specified medical care',
'K58.9': 'NOT FOUND',
'E03.8': 'Other specified hypothyroidism',
'J12.8': 'Other viral pneumonia',
'L70.0': 'Acne vulgaris',
'K29.7': 'Gastritis, unspecified',
'K05.3': 'Chronic periodontitis',
'G25.9': 'Extrapyramidal and movement disorder, unspecified',
'H65.0': 'Acute serous otitis media',
'S00.8': 'Superficial injury of other parts of head',
'K82.8': 'Other specified diseases of gallbladder',
'H10.0': 'Mucopurulent conjunctivitis',
'N76.1': 'Subacute and chronic vaginitis',
'E66.0': 'Obesity due to excess calories',
'N81.2': 'Incomplete uterovaginal prolapse',
'D24': 'Benign neoplasm of breast',
'C50.9': 'Malignant neoplasm: Breast, unspecified',
'J30.4': 'Allergic rhinitis, unspecified',
'J30.0': 'Vasomotor rhinitis',
'T15.0': 'Foreign body in cornea',
'M54.6': 'Pain in thoracic spine',
'S80.0': 'Contusion of knee',
'K21.0': 'Gastro-oesophageal reflux disease with oesophagitis',
'N86': 'Erosion and ectropion of cervix uteri',
'S90.0': 'Contusion of ankle',
'J01.0': 'Acute maxillary sinusitis',
'J31.2': 'Chronic pharyngitis',
'E04.1': 'Nontoxic single thyroid nodule',
'L24.9': 'Irritant contact dermatitis, unspecified cause',
'B35.1': 'Tinea unguium',
'Z24.0': 'Need for immunization against poliomyelitis',
'N92.6': 'Irregular menstruation, unspecified',
'K29.6': 'Other gastritis',
'N41.1': 'Chronic prostatitis',
'H35.0': 'Background retinopathy and retinal vascular changes',
'K59.0': 'Constipation',
'B18.2': 'Chronic viral hepatitis C',
'M15.9': 'Polyarthrosis, unspecified',
'Z27.1': 'Need for immunization against diphtheria-tetanus-pertussis, combined [DTP]',
'J35.2': 'Hypertrophy of adenoids',
'J34.2': 'Deviated nasal septum',
'N84.0': 'Polyp of corpus uteri',
'M19.0': 'Primary arthrosis of other joints',
'R76.1': 'Abnormal reaction to tuberculin test',
'O23.5': 'Infections of the genital tract in pregnancy',
'M13.9': 'Arthritis, unspecified',
'M77.3': 'Calcaneal spur',
'I11': 'Hypertensive heart disease',
'N60.8': 'Other benign mammary dysplasias',
'I25.9': 'Chronic ischaemic heart disease, unspecified',
'M41.1': 'Juvenile idiopathic scoliosis',
'M16.1': 'Other primary coxarthrosis',
'H52.4': 'Presbyopia',
'D23.5': 'Benign neoplasm: Skin of trunk',
'S61.0': 'Open wound of finger(s) without damage to nail',
'N81.1': 'Cystocele',
'M16.0': 'Primary coxarthrosis, bilateral',
'K80.1': 'Calculus of gallbladder with other cholecystitis',
'I67.4': 'Hypertensive encephalopathy',
'H52.5': 'Disorders of accommodation',
'G35': 'Multiple sclerosis',
'J44.8': 'Other specified chronic obstructive pulmonary disease',
'L40.0': 'Psoriasis vulgaris',
'N30.0': 'Acute cystitis',
'O20.0': 'Threatened abortion',
'K10.2': 'Inflammatory conditions of jaws',
'J06.0': 'Acute laryngopharyngitis',
'J45.0': 'Predominantly allergic asthma',
'J42': 'Unspecified chronic bronchitis',
'Z03.3': 'Observation for suspected nervous system disorder',
'N70.1': 'Chronic salpingitis and oophoritis',
'J04.0': 'Acute laryngitis',
'J41.0': 'Simple chronic bronchitis',
'G20': 'Parkinson disease',
'H65.1': 'Other acute nonsuppurative otitis media',
'K42.9': 'Umbilical hernia without obstruction or gangrene',
'G93.8': 'Other specified disorders of brain',
'F11.2': 'Mental and behavioural disorders due to use of opioids : dependence syndrome',
'M54.9': 'Dorsalgia, unspecified'}
for mkb_code, mkb_name in mkb_top_name_simple_dict.items():
if mkb_name == 'NOT FOUND':
print(mkb_code, mkb_name, sep=':')
K58.9:NOT FOUND
mkb_top_name_simple_dict['K58.9'] = mkb_top_name_dict['K58.9']
for mkb_code, mkb_name in mkb_top_name_simple_dict.items():
if mkb_name == 'NOT FOUND':
print(mkb_code, mkb_name, sep=':')
mkb_top_name_simple_dict
{'J06.9': 'Acute upper respiratory infection, unspecified',
'Z25.8': 'Need for immunization against other specified single viral diseases',
'Z00.0': 'General medical examination',
'I11.9': 'Hypertensive heart disease without (congestive) heart failure',
'K02.1': 'Caries of dentine',
'Z11.5': 'Special screening examination for other viral diseases',
'Z02.7': 'Issue of medical certificate',
'Z00.1': 'Routine child health examination',
'J02.9': 'Acute pharyngitis, unspecified',
'Z01.2': 'Dental examination',
'K04.5': 'Chronic apical periodontitis',
'J00': 'Acute nasopharyngitis [common cold]',
'M42.1': 'Adult osteochondrosis of spine',
'K04.0': 'Pulpitis',
'Z01.4': 'Gynaecological examination (general)(routine)',
'J06.8': 'Other acute upper respiratory infections of multiple sites',
'Z01.7': 'Laboratory examination',
'I67.8': 'Other specified cerebrovascular diseases',
'Z00': 'General examination and investigation of persons without complaint and reported diagnosis',
'J04.1': 'Acute tracheitis',
'Z11.3': 'Special screening examination for infections with a predominantly sexual mode of transmission',
'N60.1': 'Diffuse cystic mastopathy',
'Z01.0': 'Examination of eyes and vision',
'K04.4': 'Acute apical periodontitis of pulpal origin',
'Z20.8': 'Contact with and exposure to other communicable diseases',
'H52.1': 'Myopia',
'K02.8': 'Other dental caries',
'Z11.1': 'Special screening examination for respiratory tuberculosis',
'J20.9': 'Acute bronchitis, unspecified',
'Z01.1': 'Examination of ears and hearing',
'Z02.5': 'Examination for participation in sport',
'N95.2': 'Postmenopausal atrophic vaginitis',
'Z00.8': 'Other general examinations',
'I67.9': 'Cerebrovascular disease, unspecified',
'I67.2': 'Cerebral atherosclerosis',
'M54.4': 'Lumbago with sciatica',
'Z71.2': 'Person consulting for explanation of investigation findings',
'Z25.1': 'Need for immunization against influenza',
'O99.8': 'Other specified diseases and conditions complicating pregnancy, childbirth and the puerperium',
'G93.4': 'Encephalopathy, unspecified',
'H40.1': 'Primary open-angle glaucoma',
'G90.8': 'Other disorders of autonomic nervous system',
'Z01.8': 'Other specified special examinations',
'K59.9': 'Functional intestinal disorder, unspecified',
'K07.3': 'Anomalies of tooth position',
'I25.8': 'Other forms of chronic ischaemic heart disease',
'N40': 'Hyperplasia of prostate',
'H25.0': 'Senile incipient cataract',
'F10.2': 'Mental and behavioural disorders due to use of alcohol : dependence syndrome',
'H61.2': 'Impacted cerumen',
'G90.9': 'Disorder of autonomic nervous system, unspecified',
'Z02.0': 'Examination for admission to educational institution',
'I25.1': 'Atherosclerotic heart disease',
'M54.5': 'Low back pain',
'Z76.0': 'Issue of repeat prescription',
'K02.0': 'Caries limited to enamel',
'Z27.3': 'Need for immunization against diphtheria-tetanus-pertussis with poliomyelitis [DTP + polio]',
'J18.9': 'Pneumonia, unspecified',
'I83.9': 'Varicose veins of lower extremities without ulcer or inflammation',
'N76.0': 'Acute vaginitis',
'K07.2': 'Anomalies of dental arch relationship',
'Z27.8': 'Need for immunization against other combinations of infectious diseases',
'M95.8': 'Other specified acquired deformities of musculoskeletal system',
'M54.2': 'Cervicalgia',
'I20.8': 'Other forms of angina pectoris',
'I25.2': 'Old myocardial infarction',
'M42.9': 'Spinal osteochondrosis, unspecified',
'M17.0': 'Primary gonarthrosis, bilateral',
'K29.9': 'Gastroduodenitis, unspecified',
'C61': 'Malignant neoplasm of prostate',
'H90.3': 'Sensorineural hearing loss, bilateral',
'Z34.8': 'Supervision of other normal pregnancy',
'E06.3': 'Autoimmune thyroiditis',
'L23.9': 'Allergic contact dermatitis, unspecified cause',
'J45.8': 'Mixed asthma',
'Z03.8': 'Observation for other suspected diseases and conditions',
'B01.9': 'Varicella without complication',
'H35.3': 'Degeneration of macula and posterior pole',
'F20.0': 'Paranoid schizophrenia',
'E04.2': 'Nontoxic multinodular goitre',
'M54.1': 'Radiculopathy',
'C50.4': 'Malignant neoplasm: Upper-outer quadrant of breast',
'M17.1': 'Other primary gonarthrosis',
'H25.8': 'Other senile cataract',
'J35.0': 'Chronic tonsillitis',
'K81.1': 'Chronic cholecystitis',
'M21.4': 'Flat foot [pes planus] (acquired)',
'I70.2': 'Atherosclerosis of arteries of extremities',
'H52.0': 'Hypermetropia',
'H52.2': 'Astigmatism',
'I10': 'Essential (primary) hypertension',
'B07': 'Viral warts',
'J03.9': 'Acute tonsillitis, unspecified',
'O99.0': 'Anaemia complicating pregnancy, childbirth and the puerperium',
'S93.4': 'Sprain and strain of ankle',
'D25.1': 'Intramural leiomyoma of uterus',
'Z00.4': 'General psychiatric examination, not elsewhere classified',
'Z03.0': 'Observation for suspected tuberculosis',
'K29.5': 'Chronic gastritis, unspecified',
'M51.1': 'Lumbar and other intervertebral disc disorders with radiculopathy',
'S60.0': 'Contusion of finger(s) without damage to nail',
'N95.1': 'Menopausal and female climacteric states',
'J04.2': 'Acute laryngotracheitis',
'I11.0': 'Hypertensive heart disease with (congestive) heart failure',
'Z27.4': 'Need for immunization against measles-mumps-rubella [MMR]',
'N72': 'Inflammatory disease of cervix uteri',
'M15.0': 'Primary generalized (osteo)arthrosis',
'Z04.8': 'Examination and observation for other specified reasons',
'M21.0': 'Valgus deformity, not elsewhere classified',
'Z20.1': 'Contact with and exposure to tuberculosis',
'K86.1': 'Other chronic pancreatitis',
'O26.0': 'Excessive weight gain in pregnancy',
'J02': 'Acute pharyngitis',
'N20.0': 'Calculus of kidney',
'Z24.6': 'Need for immunization against viral hepatitis',
'Z34.0': 'Supervision of normal first pregnancy',
'S20.2': 'Contusion of thorax',
'I69.3': 'Sequelae of cerebral infarction',
'D50.9': 'Iron deficiency anaemia, unspecified',
'Z32.1': 'Pregnancy confirmed',
'S52.5': 'Fracture of lower end of radius',
'N60.9': 'Benign mammary dysplasia, unspecified',
'K00.7': 'Teething syndrome',
'Z51.8': 'Other specified medical care',
'K58.9': 'Irritable bowel syndrome without diarrhea',
'E03.8': 'Other specified hypothyroidism',
'J12.8': 'Other viral pneumonia',
'L70.0': 'Acne vulgaris',
'K29.7': 'Gastritis, unspecified',
'K05.3': 'Chronic periodontitis',
'G25.9': 'Extrapyramidal and movement disorder, unspecified',
'H65.0': 'Acute serous otitis media',
'S00.8': 'Superficial injury of other parts of head',
'K82.8': 'Other specified diseases of gallbladder',
'H10.0': 'Mucopurulent conjunctivitis',
'N76.1': 'Subacute and chronic vaginitis',
'E66.0': 'Obesity due to excess calories',
'N81.2': 'Incomplete uterovaginal prolapse',
'D24': 'Benign neoplasm of breast',
'C50.9': 'Malignant neoplasm: Breast, unspecified',
'J30.4': 'Allergic rhinitis, unspecified',
'J30.0': 'Vasomotor rhinitis',
'T15.0': 'Foreign body in cornea',
'M54.6': 'Pain in thoracic spine',
'S80.0': 'Contusion of knee',
'K21.0': 'Gastro-oesophageal reflux disease with oesophagitis',
'N86': 'Erosion and ectropion of cervix uteri',
'S90.0': 'Contusion of ankle',
'J01.0': 'Acute maxillary sinusitis',
'J31.2': 'Chronic pharyngitis',
'E04.1': 'Nontoxic single thyroid nodule',
'L24.9': 'Irritant contact dermatitis, unspecified cause',
'B35.1': 'Tinea unguium',
'Z24.0': 'Need for immunization against poliomyelitis',
'N92.6': 'Irregular menstruation, unspecified',
'K29.6': 'Other gastritis',
'N41.1': 'Chronic prostatitis',
'H35.0': 'Background retinopathy and retinal vascular changes',
'K59.0': 'Constipation',
'B18.2': 'Chronic viral hepatitis C',
'M15.9': 'Polyarthrosis, unspecified',
'Z27.1': 'Need for immunization against diphtheria-tetanus-pertussis, combined [DTP]',
'J35.2': 'Hypertrophy of adenoids',
'J34.2': 'Deviated nasal septum',
'N84.0': 'Polyp of corpus uteri',
'M19.0': 'Primary arthrosis of other joints',
'R76.1': 'Abnormal reaction to tuberculin test',
'O23.5': 'Infections of the genital tract in pregnancy',
'M13.9': 'Arthritis, unspecified',
'M77.3': 'Calcaneal spur',
'I11': 'Hypertensive heart disease',
'N60.8': 'Other benign mammary dysplasias',
'I25.9': 'Chronic ischaemic heart disease, unspecified',
'M41.1': 'Juvenile idiopathic scoliosis',
'M16.1': 'Other primary coxarthrosis',
'H52.4': 'Presbyopia',
'D23.5': 'Benign neoplasm: Skin of trunk',
'S61.0': 'Open wound of finger(s) without damage to nail',
'N81.1': 'Cystocele',
'M16.0': 'Primary coxarthrosis, bilateral',
'K80.1': 'Calculus of gallbladder with other cholecystitis',
'I67.4': 'Hypertensive encephalopathy',
'H52.5': 'Disorders of accommodation',
'G35': 'Multiple sclerosis',
'J44.8': 'Other specified chronic obstructive pulmonary disease',
'L40.0': 'Psoriasis vulgaris',
'N30.0': 'Acute cystitis',
'O20.0': 'Threatened abortion',
'K10.2': 'Inflammatory conditions of jaws',
'J06.0': 'Acute laryngopharyngitis',
'J45.0': 'Predominantly allergic asthma',
'J42': 'Unspecified chronic bronchitis',
'Z03.3': 'Observation for suspected nervous system disorder',
'N70.1': 'Chronic salpingitis and oophoritis',
'J04.0': 'Acute laryngitis',
'J41.0': 'Simple chronic bronchitis',
'G20': 'Parkinson disease',
'H65.1': 'Other acute nonsuppurative otitis media',
'K42.9': 'Umbilical hernia without obstruction or gangrene',
'G93.8': 'Other specified disorders of brain',
'F11.2': 'Mental and behavioural disorders due to use of opioids : dependence syndrome',
'M54.9': 'Dorsalgia, unspecified'}
mkb_count
| MKB_CODE | PATIENT_ID_COUNT | |
|---|---|---|
| 2847 | J06.9 | 919789 |
| 7268 | Z25.8 | 663351 |
| 7123 | Z00.0 | 611981 |
| 2506 | I11.9 | 350152 |
| 3066 | K02.1 | 316946 |
| ... | ... | ... |
| 5028 | P81.0 | 1 |
| 1478 | E76.9 | 1 |
| 1480 | E77.8 | 1 |
| 6058 | S37.6 | 1 |
| 4929 | P00.9 | 1 |
7644 rows × 2 columns
mkb_top_plot = mkb_count.query('PATIENT_ID_COUNT > 10000').loc[:, ['MKB_CODE', 'PATIENT_ID_COUNT']].reset_index(drop=True)
plt.figure(figsize=(16, 8), dpi=80)
plt.bar(mkb_top_plot['MKB_CODE'], mkb_top_plot['PATIENT_ID_COUNT'])
plt.show()
train.groupby(['ADRES', 'MKB_CODE']).value_counts()
ADRES MKB_CODE PATIENT_SEX VISIT_MONTH_YEAR AGE_CATEGORY PATIENT_ID_COUNT
Багратионовск A02.0 0 12.20 elderly 1 1
A02.9 0 10.19 children 1 1
A04.3 0 02.22 middleage 1 1
A04.9 1 06.21 young 1 1
03.18 young 1 1
..
Ясное Z45 1 04.18 middleage 1 1
Z54.0 0 01.19 young 1 1
Z71.2 1 04.19 young 1 1
Z71.8 0 07.18 centenarians 1 1
1 07.18 middleage 1 1
Length: 2212393, dtype: int64
top_mkb = train.query('ADRES == "Калининград" & MKB_CODE == "J06.9"')
top_mkb
| PATIENT_SEX | MKB_CODE | ADRES | VISIT_MONTH_YEAR | AGE_CATEGORY | PATIENT_ID_COUNT | month_year | |
|---|---|---|---|---|---|---|---|
| 509571 | 0 | J06.9 | Калининград | 01.18 | centenarians | 12 | 2018-01-01 |
| 509572 | 0 | J06.9 | Калининград | 01.18 | children | 3889 | 2018-01-01 |
| 509573 | 0 | J06.9 | Калининград | 01.18 | elderly | 507 | 2018-01-01 |
| 509574 | 0 | J06.9 | Калининград | 01.18 | middleage | 575 | 2018-01-01 |
| 509575 | 0 | J06.9 | Калининград | 01.18 | old | 153 | 2018-01-01 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 1671175 | 1 | J06.9 | Калининград | 12.21 | children | 9169 | 2021-12-01 |
| 1671176 | 1 | J06.9 | Калининград | 12.21 | elderly | 531 | 2021-12-01 |
| 1671177 | 1 | J06.9 | Калининград | 12.21 | middleage | 795 | 2021-12-01 |
| 1671178 | 1 | J06.9 | Калининград | 12.21 | old | 78 | 2021-12-01 |
| 1671179 | 1 | J06.9 | Калининград | 12.21 | young | 2638 | 2021-12-01 |
612 rows × 7 columns
top_mkb_f = top_mkb.query('PATIENT_SEX == "0"')
top_mkb_f
| PATIENT_SEX | MKB_CODE | ADRES | VISIT_MONTH_YEAR | AGE_CATEGORY | PATIENT_ID_COUNT | month_year | |
|---|---|---|---|---|---|---|---|
| 509571 | 0 | J06.9 | Калининград | 01.18 | centenarians | 12 | 2018-01-01 |
| 509572 | 0 | J06.9 | Калининград | 01.18 | children | 3889 | 2018-01-01 |
| 509573 | 0 | J06.9 | Калининград | 01.18 | elderly | 507 | 2018-01-01 |
| 509574 | 0 | J06.9 | Калининград | 01.18 | middleage | 575 | 2018-01-01 |
| 509575 | 0 | J06.9 | Калининград | 01.18 | old | 153 | 2018-01-01 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 509872 | 0 | J06.9 | Калининград | 12.21 | children | 8031 | 2021-12-01 |
| 509873 | 0 | J06.9 | Калининград | 12.21 | elderly | 1093 | 2021-12-01 |
| 509874 | 0 | J06.9 | Калининград | 12.21 | middleage | 1381 | 2021-12-01 |
| 509875 | 0 | J06.9 | Калининград | 12.21 | old | 220 | 2021-12-01 |
| 509876 | 0 | J06.9 | Калининград | 12.21 | young | 3792 | 2021-12-01 |
306 rows × 7 columns
top_mkb_m = top_mkb.query('PATIENT_SEX == "1"')
top_mkb_m
| PATIENT_SEX | MKB_CODE | ADRES | VISIT_MONTH_YEAR | AGE_CATEGORY | PATIENT_ID_COUNT | month_year | |
|---|---|---|---|---|---|---|---|
| 1670874 | 1 | J06.9 | Калининград | 01.18 | centenarians | 10 | 2018-01-01 |
| 1670875 | 1 | J06.9 | Калининград | 01.18 | children | 4129 | 2018-01-01 |
| 1670876 | 1 | J06.9 | Калининград | 01.18 | elderly | 198 | 2018-01-01 |
| 1670877 | 1 | J06.9 | Калининград | 01.18 | middleage | 331 | 2018-01-01 |
| 1670878 | 1 | J06.9 | Калининград | 01.18 | old | 39 | 2018-01-01 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 1671175 | 1 | J06.9 | Калининград | 12.21 | children | 9169 | 2021-12-01 |
| 1671176 | 1 | J06.9 | Калининград | 12.21 | elderly | 531 | 2021-12-01 |
| 1671177 | 1 | J06.9 | Калининград | 12.21 | middleage | 795 | 2021-12-01 |
| 1671178 | 1 | J06.9 | Калининград | 12.21 | old | 78 | 2021-12-01 |
| 1671179 | 1 | J06.9 | Калининград | 12.21 | young | 2638 | 2021-12-01 |
306 rows × 7 columns
list(range(2018, 2023))
[2018, 2019, 2020, 2021, 2022]
import datetime as dt
plt.figure(figsize=(12, 6), dpi=80)
plt.plot(top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT'), marker = '+')
plt.plot(top_mkb_f.groupby('month_year').sum('PATIENT_ID_COUNT'), marker = 'v')
plt.plot(top_mkb_m.groupby('month_year').sum('PATIENT_ID_COUNT'), marker = '^')
[plt.axvline([dt.datetime(year, 1, 1)]) for year in range(2018, 2023)]
plt.grid()
plt.show()
plt.figure(figsize=(12, 6), dpi=80)
top_mkb_by_age = top_mkb.groupby('AGE_CATEGORY').groups.keys()
for age_group in top_mkb_by_age:
age_group_filtered = top_mkb.query('AGE_CATEGORY == @age_group').sort_values(by='month_year', ascending=True)
plt.plot(age_group_filtered.loc[:, 'month_year'], age_group_filtered.loc[:, 'PATIENT_ID_COUNT'], marker = 'o')
[plt.axvline([dt.datetime(year, 1, 1)]) for year in range(2018, 2023)]
plt.legend(list(top_mkb_by_age))
plt.grid()
plt.show()
# Moving Average
top_mkb_total = top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT')
#print(top_mkb_total)
moving_average = top_mkb_total.rolling(
window=12, # 365-day window
center=True, # puts the average at the center of the window
min_periods=6, # choose about half the window size
).mean() # compute the mean (could also do median, std, min, max, ...)
ax = top_mkb_total.plot(figsize=(12, 6), style=".", color="0.5", grid=True)
moving_average.plot(
ax=ax, linewidth=3, title="Top MKB Total Moving Average", legend=False,
);
[plt.axvline([dt.datetime(year, 1, 1)]) for year in range(2018, 2023)]
plt.grid()
plt.show()
import datetime as dt
# Coranavirus turning trend point at 2020-05-01
corona_turn = dt.datetime(2020, 5, 1)
corona_turn_idx = top_mkb_total[:corona_turn].shape[0]
top_mkb_total.index[0:corona_turn_idx]
DatetimeIndex(['2018-01-01', '2018-02-01', '2018-03-01', '2018-04-01',
'2018-05-01', '2018-06-01', '2018-07-01', '2018-08-01',
'2018-09-01', '2018-10-01', '2018-11-01', '2018-12-01',
'2019-01-01', '2019-02-01', '2019-03-01', '2019-04-01',
'2019-05-01', '2019-06-01', '2019-07-01', '2019-08-01',
'2019-09-01', '2019-10-01', '2019-11-01', '2019-12-01',
'2020-01-01', '2020-02-01', '2020-03-01', '2020-04-01',
'2020-05-01'],
dtype='datetime64[ns]', name='month_year', freq=None)
from statsmodels.tsa.deterministic import DeterministicProcess
dp_prior = DeterministicProcess(
index=top_mkb_total.index[:corona_turn_idx], # dates from the training data
constant=True, # dummy feature for the bias (y_intercept)
order=1, # the time dummy (trend)
drop=True, # drop terms if necessary to avoid collinearity
)
dp_after = DeterministicProcess(
index=top_mkb_total.index[corona_turn_idx-1:], # dates from the training data
constant=True, # dummy feature for the bias (y_intercept)
order=1, # the time dummy (trend)
drop=True, # drop terms if necessary to avoid collinearity
)
# `in_sample` creates features for the dates given in the `index` argument
X_prior = dp_prior.in_sample()
X_after = dp_after.in_sample()
print(X_prior.tail())
print(X_after.head())
const trend
month_year
2020-01-01 1.0 25.0
2020-02-01 1.0 26.0
2020-03-01 1.0 27.0
2020-04-01 1.0 28.0
2020-05-01 1.0 29.0
const trend
month_year
2020-05-01 1.0 1.0
2020-06-01 1.0 2.0
2020-07-01 1.0 3.0
2020-08-01 1.0 4.0
2020-09-01 1.0 5.0
dp_after.out_of_sample(1)
| const | trend | |
|---|---|---|
| 24 | 1.0 | 24.0 |
from sklearn.linear_model import LinearRegression
y_prior = top_mkb_total['PATIENT_ID_COUNT'][:corona_turn_idx] # the target
y_after = top_mkb_total['PATIENT_ID_COUNT'][corona_turn_idx-1:] # the target
# The intercept is the same as the `const` feature from
# DeterministicProcess. LinearRegression behaves badly with duplicated
# features, so we need to be sure to exclude it here.
model_prior = LinearRegression(fit_intercept=False)
model_prior.fit(X_prior, y_prior)
model_after = LinearRegression(fit_intercept=False)
model_after.fit(X_after, y_after)
y_pred_prior = pd.Series(model_prior.predict(X_prior), index=X_prior.index)
y_pred_after = pd.Series(model_after.predict(X_after), index=X_after.index)
y_pred_total = pd.concat([y_pred_prior, y_pred_after], axis=0)
print(y_pred_total.reset_index(drop=True).tail())
47 16937.640316 48 17478.371542 49 18019.102767 50 18559.833992 51 19100.565217 dtype: float64
X_prior.index
DatetimeIndex(['2018-01-01', '2018-02-01', '2018-03-01', '2018-04-01',
'2018-05-01', '2018-06-01', '2018-07-01', '2018-08-01',
'2018-09-01', '2018-10-01', '2018-11-01', '2018-12-01',
'2019-01-01', '2019-02-01', '2019-03-01', '2019-04-01',
'2019-05-01', '2019-06-01', '2019-07-01', '2019-08-01',
'2019-09-01', '2019-10-01', '2019-11-01', '2019-12-01',
'2020-01-01', '2020-02-01', '2020-03-01', '2020-04-01',
'2020-05-01'],
dtype='datetime64[ns]', name='month_year', freq=None)
plt.figure(figsize=(12, 6), dpi=80)
plt.plot(X_prior.index, y_pred_prior, marker = 'o')
plt.plot(X_after.index, y_pred_after, marker = 'o')
plt.plot(top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT'), marker = '+')
plt.plot(top_mkb_f.groupby('month_year').sum('PATIENT_ID_COUNT'), marker = 'v')
plt.plot(top_mkb_m.groupby('month_year').sum('PATIENT_ID_COUNT'), marker = '^')
[plt.axvline([dt.datetime(year, 1, 1)]) for year in range(2018, 2023)]
plt.legend(['fit before corona', 'fit after corona', 'total', 'female', 'male'])
plt.grid()
plt.show()
import rsnd
fft_ = rsnd.fft_angle(top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT'), 1)
fft_
{'frequency': array([0. , 0.01960784, 0.03921569, 0.05882353, 0.07843137,
0.09803922, 0.11764706, 0.1372549 , 0.15686275, 0.17647059,
0.19607843, 0.21568627, 0.23529412, 0.25490196, 0.2745098 ,
0.29411765, 0.31372549, 0.33333333, 0.35294118, 0.37254902,
0.39215686, 0.41176471, 0.43137255, 0.45098039, 0.47058824]),
'fft_amplitude': array([[236.96078431],
[621.37254902],
[733.45098039],
[609.29411765],
[319.45098039],
[232.2745098 ],
[221.64705882],
[283.60784314],
[465.64705882],
[546.74509804],
[478.31372549],
[497.80392157],
[526.58823529],
[887.29411765],
[556.54901961],
[431.29411765],
[262.62745098],
[216.31372549],
[231.76470588],
[244.31372549],
[495.80392157],
[522.43137255],
[384.31372549],
[454.43137255],
[408.62745098]])}
plt.plot(1/fft_['frequency'], fft_['fft_amplitude'], marker='o')
plt.show()
1/fft_['frequency']
array([ inf, 51. , 25.5 , 17. , 12.75 ,
10.2 , 8.5 , 7.28571429, 6.375 , 5.66666667,
5.1 , 4.63636364, 4.25 , 3.92307692, 3.64285714,
3.4 , 3.1875 , 3. , 2.83333333, 2.68421053,
2.55 , 2.42857143, 2.31818182, 2.2173913 , 2.125 ])
1/fft_['frequency'][11]
4.636363636363636
fft_['fft_amplitude']
array([[236.96078431],
[621.37254902],
[733.45098039],
[609.29411765],
[319.45098039],
[232.2745098 ],
[221.64705882],
[283.60784314],
[465.64705882],
[546.74509804],
[478.31372549],
[497.80392157],
[526.58823529],
[887.29411765],
[556.54901961],
[431.29411765],
[262.62745098],
[216.31372549],
[231.76470588],
[244.31372549],
[495.80392157],
[522.43137255],
[384.31372549],
[454.43137255],
[408.62745098]])
# @link https://stackoverflow.com/questions/27516849/how-to-convert-list-of-numpy-arrays-into-single-numpy-array
np.concatenate(fft_['fft_amplitude'], axis=0)
array([236.96078431, 621.37254902, 733.45098039, 609.29411765,
319.45098039, 232.2745098 , 221.64705882, 283.60784314,
465.64705882, 546.74509804, 478.31372549, 497.80392157,
526.58823529, 887.29411765, 556.54901961, 431.29411765,
262.62745098, 216.31372549, 231.76470588, 244.31372549,
495.80392157, 522.43137255, 384.31372549, 454.43137255,
408.62745098])
np.stack(fft_['fft_amplitude'], axis=0)
np.vstack(fft_['fft_amplitude'])
np.array(fft_['fft_amplitude'])
array([[236.96078431],
[621.37254902],
[733.45098039],
[609.29411765],
[319.45098039],
[232.2745098 ],
[221.64705882],
[283.60784314],
[465.64705882],
[546.74509804],
[478.31372549],
[497.80392157],
[526.58823529],
[887.29411765],
[556.54901961],
[431.29411765],
[262.62745098],
[216.31372549],
[231.76470588],
[244.31372549],
[495.80392157],
[522.43137255],
[384.31372549],
[454.43137255],
[408.62745098]])
s1 = pd.Series(1/fft_['frequency'])
s2 = pd.Series(np.concatenate(fft_['fft_amplitude'], axis=0))
f=pd.concat([s1, s2], axis = 1, names=['Period, month', 'Amplitude'])
Раз в 3 месяца пик самый большой.
plt.plot(f[0], f[1], marker='o')
plt.show()
y_p_t = pd.DataFrame(y_pred_total)
y_p_t.rename(columns={0: 'PATIENT_ID_COUNT'}, inplace=True)
y_p_t
| PATIENT_ID_COUNT | |
|---|---|
| month_year | |
| 2018-01-01 | 12805.462069 |
| 2018-02-01 | 12691.392118 |
| 2018-03-01 | 12577.322167 |
| 2018-04-01 | 12463.252217 |
| 2018-05-01 | 12349.182266 |
| 2018-06-01 | 12235.112315 |
| 2018-07-01 | 12121.042365 |
| 2018-08-01 | 12006.972414 |
| 2018-09-01 | 11892.902463 |
| 2018-10-01 | 11778.832512 |
| 2018-11-01 | 11664.762562 |
| 2018-12-01 | 11550.692611 |
| 2019-01-01 | 11436.622660 |
| 2019-02-01 | 11322.552709 |
| 2019-03-01 | 11208.482759 |
| 2019-04-01 | 11094.412808 |
| 2019-05-01 | 10980.342857 |
| 2019-06-01 | 10866.272906 |
| 2019-07-01 | 10752.202956 |
| 2019-08-01 | 10638.133005 |
| 2019-09-01 | 10524.063054 |
| 2019-10-01 | 10409.993103 |
| 2019-11-01 | 10295.923153 |
| 2019-12-01 | 10181.853202 |
| 2020-01-01 | 10067.783251 |
| 2020-02-01 | 9953.713300 |
| 2020-03-01 | 9839.643350 |
| 2020-04-01 | 9725.573399 |
| 2020-05-01 | 9611.503448 |
| 2020-05-01 | 7204.478261 |
| 2020-06-01 | 7745.209486 |
| 2020-07-01 | 8285.940711 |
| 2020-08-01 | 8826.671937 |
| 2020-09-01 | 9367.403162 |
| 2020-10-01 | 9908.134387 |
| 2020-11-01 | 10448.865613 |
| 2020-12-01 | 10989.596838 |
| 2021-01-01 | 11530.328063 |
| 2021-02-01 | 12071.059289 |
| 2021-03-01 | 12611.790514 |
| 2021-04-01 | 13152.521739 |
| 2021-05-01 | 13693.252964 |
| 2021-06-01 | 14233.984190 |
| 2021-07-01 | 14774.715415 |
| 2021-08-01 | 15315.446640 |
| 2021-09-01 | 15856.177866 |
| 2021-10-01 | 16396.909091 |
| 2021-11-01 | 16937.640316 |
| 2021-12-01 | 17478.371542 |
| 2022-01-01 | 18019.102767 |
| 2022-02-01 | 18559.833992 |
| 2022-03-01 | 19100.565217 |
top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT') - y_p_t
| PATIENT_ID_COUNT | |
|---|---|
| month_year | |
| 2018-01-01 | -720.462069 |
| 2018-02-01 | 3153.607882 |
| 2018-03-01 | 6125.677833 |
| 2018-04-01 | 3073.747783 |
| 2018-05-01 | -4203.182266 |
| 2018-06-01 | -6312.112315 |
| 2018-07-01 | -6469.042365 |
| 2018-08-01 | -4774.972414 |
| 2018-09-01 | -18.902463 |
| 2018-10-01 | 2163.167488 |
| 2018-11-01 | 532.237438 |
| 2018-12-01 | 1143.307389 |
| 2019-01-01 | 1991.377340 |
| 2019-02-01 | 11303.447291 |
| 2019-03-01 | 2983.517241 |
| 2019-04-01 | -96.412808 |
| 2019-05-01 | -4283.342857 |
| 2019-06-01 | -5350.272906 |
| 2019-07-01 | -4842.202956 |
| 2019-08-01 | -4408.133005 |
| 2019-09-01 | 2118.936946 |
| 2019-10-01 | 2912.006897 |
| 2019-11-01 | -495.923153 |
| 2019-12-01 | 1406.146798 |
| 2020-01-01 | 352.216749 |
| 2020-02-01 | 7325.286700 |
| 2020-03-01 | 6853.356650 |
| 2020-04-01 | -4239.573399 |
| 2020-05-01 | -7223.503448 |
| 2020-05-01 | -4816.478261 |
| 2020-06-01 | -5271.209486 |
| 2020-07-01 | -2594.940711 |
| 2020-08-01 | -3375.671937 |
| 2020-09-01 | 1934.596838 |
| 2020-10-01 | 5997.865613 |
| 2020-11-01 | 11603.134387 |
| 2020-12-01 | 11916.403162 |
| 2021-01-01 | -1784.328063 |
| 2021-02-01 | -970.059289 |
| 2021-03-01 | -1721.790514 |
| 2021-04-01 | -2534.521739 |
| 2021-05-01 | -5117.252964 |
| 2021-06-01 | -5499.984190 |
| 2021-07-01 | -4734.715415 |
| 2021-08-01 | -4236.446640 |
| 2021-09-01 | 5188.822134 |
| 2021-10-01 | 6913.090909 |
| 2021-11-01 | -3496.640316 |
| 2021-12-01 | 10336.628458 |
| 2022-01-01 | 352.897233 |
| 2022-02-01 | 2422.166008 |
| 2022-03-01 | -10511.565217 |
plt.figure(figsize=(12, 6), dpi=80)
plt.plot(top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT') - y_p_t, marker = '+')
plt.plot(top_mkb_f.groupby('month_year').sum('PATIENT_ID_COUNT') - y_p_t, marker = 'v')
plt.plot(top_mkb_m.groupby('month_year').sum('PATIENT_ID_COUNT') - y_p_t, marker = '^')
[plt.axvline([dt.datetime(year, 1, 1)]) for year in range(2018, 2023)]
plt.title('DETRENDED')
plt.legend(['total', 'female', 'male'])
plt.grid()
plt.show()
# @link https://www.kaggle.com/code/ryanholbrook/hybrid-models/tutorial
y_detrend = top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT') - y_p_t
y_detrend
| PATIENT_ID_COUNT | |
|---|---|
| month_year | |
| 2018-01-01 | -720.462069 |
| 2018-02-01 | 3153.607882 |
| 2018-03-01 | 6125.677833 |
| 2018-04-01 | 3073.747783 |
| 2018-05-01 | -4203.182266 |
| 2018-06-01 | -6312.112315 |
| 2018-07-01 | -6469.042365 |
| 2018-08-01 | -4774.972414 |
| 2018-09-01 | -18.902463 |
| 2018-10-01 | 2163.167488 |
| 2018-11-01 | 532.237438 |
| 2018-12-01 | 1143.307389 |
| 2019-01-01 | 1991.377340 |
| 2019-02-01 | 11303.447291 |
| 2019-03-01 | 2983.517241 |
| 2019-04-01 | -96.412808 |
| 2019-05-01 | -4283.342857 |
| 2019-06-01 | -5350.272906 |
| 2019-07-01 | -4842.202956 |
| 2019-08-01 | -4408.133005 |
| 2019-09-01 | 2118.936946 |
| 2019-10-01 | 2912.006897 |
| 2019-11-01 | -495.923153 |
| 2019-12-01 | 1406.146798 |
| 2020-01-01 | 352.216749 |
| 2020-02-01 | 7325.286700 |
| 2020-03-01 | 6853.356650 |
| 2020-04-01 | -4239.573399 |
| 2020-05-01 | -7223.503448 |
| 2020-05-01 | -4816.478261 |
| 2020-06-01 | -5271.209486 |
| 2020-07-01 | -2594.940711 |
| 2020-08-01 | -3375.671937 |
| 2020-09-01 | 1934.596838 |
| 2020-10-01 | 5997.865613 |
| 2020-11-01 | 11603.134387 |
| 2020-12-01 | 11916.403162 |
| 2021-01-01 | -1784.328063 |
| 2021-02-01 | -970.059289 |
| 2021-03-01 | -1721.790514 |
| 2021-04-01 | -2534.521739 |
| 2021-05-01 | -5117.252964 |
| 2021-06-01 | -5499.984190 |
| 2021-07-01 | -4734.715415 |
| 2021-08-01 | -4236.446640 |
| 2021-09-01 | 5188.822134 |
| 2021-10-01 | 6913.090909 |
| 2021-11-01 | -3496.640316 |
| 2021-12-01 | 10336.628458 |
| 2022-01-01 | 352.897233 |
| 2022-02-01 | 2422.166008 |
| 2022-03-01 | -10511.565217 |
from scipy import integrate
#y_detrend.apply(lambda g: integrate.trapz(g.PATIENT_ID_COUNT, x=g.month_year))
y_detrend[:52].sum()
PATIENT_ID_COUNT 3.637979e-11 dtype: float64
y_p_t
| PATIENT_ID_COUNT | |
|---|---|
| month_year | |
| 2018-01-01 | 12805.462069 |
| 2018-02-01 | 12691.392118 |
| 2018-03-01 | 12577.322167 |
| 2018-04-01 | 12463.252217 |
| 2018-05-01 | 12349.182266 |
| 2018-06-01 | 12235.112315 |
| 2018-07-01 | 12121.042365 |
| 2018-08-01 | 12006.972414 |
| 2018-09-01 | 11892.902463 |
| 2018-10-01 | 11778.832512 |
| 2018-11-01 | 11664.762562 |
| 2018-12-01 | 11550.692611 |
| 2019-01-01 | 11436.622660 |
| 2019-02-01 | 11322.552709 |
| 2019-03-01 | 11208.482759 |
| 2019-04-01 | 11094.412808 |
| 2019-05-01 | 10980.342857 |
| 2019-06-01 | 10866.272906 |
| 2019-07-01 | 10752.202956 |
| 2019-08-01 | 10638.133005 |
| 2019-09-01 | 10524.063054 |
| 2019-10-01 | 10409.993103 |
| 2019-11-01 | 10295.923153 |
| 2019-12-01 | 10181.853202 |
| 2020-01-01 | 10067.783251 |
| 2020-02-01 | 9953.713300 |
| 2020-03-01 | 9839.643350 |
| 2020-04-01 | 9725.573399 |
| 2020-05-01 | 9611.503448 |
| 2020-05-01 | 7204.478261 |
| 2020-06-01 | 7745.209486 |
| 2020-07-01 | 8285.940711 |
| 2020-08-01 | 8826.671937 |
| 2020-09-01 | 9367.403162 |
| 2020-10-01 | 9908.134387 |
| 2020-11-01 | 10448.865613 |
| 2020-12-01 | 10989.596838 |
| 2021-01-01 | 11530.328063 |
| 2021-02-01 | 12071.059289 |
| 2021-03-01 | 12611.790514 |
| 2021-04-01 | 13152.521739 |
| 2021-05-01 | 13693.252964 |
| 2021-06-01 | 14233.984190 |
| 2021-07-01 | 14774.715415 |
| 2021-08-01 | 15315.446640 |
| 2021-09-01 | 15856.177866 |
| 2021-10-01 | 16396.909091 |
| 2021-11-01 | 16937.640316 |
| 2021-12-01 | 17478.371542 |
| 2022-01-01 | 18019.102767 |
| 2022-02-01 | 18559.833992 |
| 2022-03-01 | 19100.565217 |
#model_after.predict(X)
#y_pred_after = pd.Series(model_after.predict(X_after), index=X_after.index)
X_after
X_after.index[-1:]
d = {'a': 1, 'b': 2, 'c': 3}
da = {dt.datetime(2022,4,1): 0}
ser = pd.Series(da, name='for pred')
#ser = pd.Series(data=d, index=['a', 'b', 'c'])
ser
2022-04-01 0 Name: for pred, dtype: int64
X
| y_lag_1 | |
|---|---|
| month_year | |
| 2018-01 | 0.0 |
| 2018-02 | 12085.0 |
| 2018-03 | 15845.0 |
| 2018-04 | 18703.0 |
| 2018-05 | 15537.0 |
| 2018-06 | 8146.0 |
| 2018-07 | 5923.0 |
| 2018-08 | 5652.0 |
| 2018-09 | 7232.0 |
| 2018-10 | 11874.0 |
| 2018-11 | 13942.0 |
| 2018-12 | 12197.0 |
| 2019-01 | 12694.0 |
| 2019-02 | 13428.0 |
| 2019-03 | 22626.0 |
| 2019-04 | 14192.0 |
| 2019-05 | 10998.0 |
| 2019-06 | 6697.0 |
| 2019-07 | 5516.0 |
| 2019-08 | 5910.0 |
| 2019-09 | 6230.0 |
| 2019-10 | 12643.0 |
| 2019-11 | 13322.0 |
| 2019-12 | 9800.0 |
| 2020-01 | 11588.0 |
| 2020-02 | 10420.0 |
| 2020-03 | 17279.0 |
| 2020-04 | 16693.0 |
| 2020-05 | 5486.0 |
| 2020-06 | 2388.0 |
| 2020-07 | 2474.0 |
| 2020-08 | 5691.0 |
| 2020-09 | 5451.0 |
| 2020-10 | 11302.0 |
| 2020-11 | 15906.0 |
| 2020-12 | 22052.0 |
| 2021-01 | 22906.0 |
| 2021-02 | 9746.0 |
| 2021-03 | 11101.0 |
| 2021-04 | 10890.0 |
| 2021-05 | 10618.0 |
| 2021-06 | 8576.0 |
| 2021-07 | 8734.0 |
| 2021-08 | 10040.0 |
| 2021-09 | 11079.0 |
| 2021-10 | 21045.0 |
| 2021-11 | 23310.0 |
| 2021-12 | 13441.0 |
| 2022-01 | 27815.0 |
| 2022-02 | 18372.0 |
| 2022-03 | 20982.0 |
!pip install xgboost
Requirement already satisfied: xgboost in d:\programdata\anaconda3\lib\site-packages (1.5.2)
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Requirement already satisfied: scipy in d:\programdata\anaconda3\lib\site-packages (from xgboost) (1.7.3) Requirement already satisfied: numpy in d:\programdata\anaconda3\lib\site-packages (from xgboost) (1.20.3)
from xgboost import XGBRegressor
# Pivot wide to long (stack) and convert DataFrame to Series (squeeze)
#y_fit = y_fit.stack().squeeze() # trend from training set
#y_pred = y_pred.stack().squeeze() # trend from test set
# Create residuals (the collection of detrended series) from the training set
#y_resid = y_train - y_fit
# Train XGBoost on the residuals
xgb = XGBRegressor()
xgb.fit(X_after, y_detrend[corona_turn_idx:])
# Add the predicted residuals onto the predicted trends
#y_fit_boosted = xgb.predict(X_train) + y_fit
#y_pred_boosted = xgb.predict(X_test) + y_pred
XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=1,
colsample_bynode=1, colsample_bytree=1, enable_categorical=False,
gamma=0, gpu_id=-1, importance_type=None,
interaction_constraints='', learning_rate=0.300000012,
max_delta_step=0, max_depth=6, min_child_weight=1, missing=nan,
monotone_constraints='()', n_estimators=100, n_jobs=8,
num_parallel_tree=1, predictor='auto', random_state=0, reg_alpha=0,
reg_lambda=1, scale_pos_weight=1, subsample=1, tree_method='exact',
validate_parameters=1, verbosity=None)
X_after
| const | trend | |
|---|---|---|
| month_year | ||
| 2020-05-01 | 1.0 | 1.0 |
| 2020-06-01 | 1.0 | 2.0 |
| 2020-07-01 | 1.0 | 3.0 |
| 2020-08-01 | 1.0 | 4.0 |
| 2020-09-01 | 1.0 | 5.0 |
| 2020-10-01 | 1.0 | 6.0 |
| 2020-11-01 | 1.0 | 7.0 |
| 2020-12-01 | 1.0 | 8.0 |
| 2021-01-01 | 1.0 | 9.0 |
| 2021-02-01 | 1.0 | 10.0 |
| 2021-03-01 | 1.0 | 11.0 |
| 2021-04-01 | 1.0 | 12.0 |
| 2021-05-01 | 1.0 | 13.0 |
| 2021-06-01 | 1.0 | 14.0 |
| 2021-07-01 | 1.0 | 15.0 |
| 2021-08-01 | 1.0 | 16.0 |
| 2021-09-01 | 1.0 | 17.0 |
| 2021-10-01 | 1.0 | 18.0 |
| 2021-11-01 | 1.0 | 19.0 |
| 2021-12-01 | 1.0 | 20.0 |
| 2022-01-01 | 1.0 | 21.0 |
| 2022-02-01 | 1.0 | 22.0 |
| 2022-03-01 | 1.0 | 23.0 |
xgb_sh_1 = XGBRegressor()
xgb_sh_1.fit(X_after[:-1], y_detrend.iloc[corona_turn_idx:-1])
XGBRegressor(base_score=0.5, booster='gbtree', colsample_bylevel=1,
colsample_bynode=1, colsample_bytree=1, enable_categorical=False,
gamma=0, gpu_id=-1, importance_type=None,
interaction_constraints='', learning_rate=0.300000012,
max_delta_step=0, max_depth=6, min_child_weight=1, missing=nan,
monotone_constraints='()', n_estimators=100, n_jobs=8,
num_parallel_tree=1, predictor='auto', random_state=0, reg_alpha=0,
reg_lambda=1, scale_pos_weight=1, subsample=1, tree_method='exact',
validate_parameters=1, verbosity=None)
X_after[-1:]
| const | trend | |
|---|---|---|
| month_year | ||
| 2022-03-01 | 1.0 | 23.0 |
y_p_t[-1:]
| PATIENT_ID_COUNT | |
|---|---|
| month_year | |
| 2022-03-01 | 19100.565217 |
xgb_sh_1.predict(X_after[-1:])
array([2422.165], dtype=float32)
y_p_t[-1:]
| PATIENT_ID_COUNT | |
|---|---|
| month_year | |
| 2022-03-01 | 19100.565217 |
xgb_sh_1.predict(X_after[-1:]) + y_p_t[-1:]
| PATIENT_ID_COUNT | |
|---|---|
| month_year | |
| 2022-03-01 | 21522.730256 |
X_test = pd.DataFrame(data = {'PATIENT_ID_COUNT': [0]}, index=[dt.datetime(2022,4,1)])
X_test
| PATIENT_ID_COUNT | |
|---|---|
| 2022-04-01 | 0 |
xgb.predict(dp_after.out_of_sample(1))
array([-10511.369], dtype=float32)
# @ 2022-04-01
xgb.predict(dp_after.out_of_sample(1))+model_after.predict(dp_after.out_of_sample(1))
array([9129.92730206])
X_test.index
DatetimeIndex(['2022-04-01'], dtype='datetime64[ns]', freq=None)
X_after
| const | trend | |
|---|---|---|
| month_year | ||
| 2020-05-01 | 1.0 | 1.0 |
| 2020-06-01 | 1.0 | 2.0 |
| 2020-07-01 | 1.0 | 3.0 |
| 2020-08-01 | 1.0 | 4.0 |
| 2020-09-01 | 1.0 | 5.0 |
| 2020-10-01 | 1.0 | 6.0 |
| 2020-11-01 | 1.0 | 7.0 |
| 2020-12-01 | 1.0 | 8.0 |
| 2021-01-01 | 1.0 | 9.0 |
| 2021-02-01 | 1.0 | 10.0 |
| 2021-03-01 | 1.0 | 11.0 |
| 2021-04-01 | 1.0 | 12.0 |
| 2021-05-01 | 1.0 | 13.0 |
| 2021-06-01 | 1.0 | 14.0 |
| 2021-07-01 | 1.0 | 15.0 |
| 2021-08-01 | 1.0 | 16.0 |
| 2021-09-01 | 1.0 | 17.0 |
| 2021-10-01 | 1.0 | 18.0 |
| 2021-11-01 | 1.0 | 19.0 |
| 2021-12-01 | 1.0 | 20.0 |
| 2022-01-01 | 1.0 | 21.0 |
| 2022-02-01 | 1.0 | 22.0 |
| 2022-03-01 | 1.0 | 23.0 |
# Add the predicted residuals onto the predicted trends
y_fit_boosted = xgb.predict(X_train) + y_fit
y_pred_boosted = xgb.predict(X_test) + y_pred
# source: https://www.kaggle.com/code/ryanholbrook/seasonality/tutorial
from pathlib import Path
from warnings import simplefilter
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.linear_model import LinearRegression
from statsmodels.tsa.deterministic import CalendarFourier, DeterministicProcess
simplefilter("ignore")
# Set Matplotlib defaults
plt.style.use("seaborn-whitegrid")
plt.rc("figure", autolayout=True, figsize=(11, 5))
plt.rc(
"axes",
labelweight="bold",
labelsize="large",
titleweight="bold",
titlesize=16,
titlepad=10,
)
plot_params = dict(
color="0.75",
style=".-",
markeredgecolor="0.25",
markerfacecolor="0.25",
legend=False,
)
%config InlineBackend.figure_format = 'retina'
# annotations: https://stackoverflow.com/a/49238256/5769929
def seasonal_plot(X, y, period, freq, ax=None):
if ax is None:
_, ax = plt.subplots()
palette = sns.color_palette("husl", n_colors=X[period].nunique(),)
ax = sns.lineplot(
x=freq,
y=y,
hue=period,
data=X,
ci=False,
ax=ax,
palette=palette,
legend=False,
)
ax.set_title(f"Seasonal Plot ({period}/{freq})")
for line, name in zip(ax.lines, X[period].unique()):
y_ = line.get_ydata()[-1]
ax.annotate(
name,
xy=(1, y_),
xytext=(6, 0),
color=line.get_color(),
xycoords=ax.get_yaxis_transform(),
textcoords="offset points",
size=14,
va="center",
)
return ax
def plot_periodogram(ts, detrend='linear', ax=None):
from scipy.signal import periodogram
fs = pd.Timedelta("1Y") / pd.Timedelta(np.timedelta64(1, "M"))
freqencies, spectrum = periodogram(
ts,
fs=fs,
detrend=detrend,
window="boxcar",
scaling='spectrum',
)
if ax is None:
_, ax = plt.subplots()
ax.step(freqencies, spectrum, color="purple")
ax.set_xscale("log")
#ax.set_xticks([1, 2, 4, 6, 12, 26, 52, 104])
ax.set_xticks([1, 2, 4, 6])
ax.set_xticklabels(
[
"Annual (1)",
"Semiannual (2)",
"Quarterly (4)",
"Bimonthly (6)",
#"Monthly (12)",
#"Biweekly (26)",
#"Weekly (52)",
#"Semiweekly (104)",
],
rotation=30,
)
ax.ticklabel_format(axis="y", style="sci", scilimits=(0, 0))
ax.set_ylabel("Variance")
ax.set_title("Periodogram")
return ax
#data_dir = Path("../input/ts-course-data")
#tunnel = pd.read_csv(data_dir / "tunnel.csv", parse_dates=["Day"])
#tunnel = tunnel.set_index("Day").to_period("D")
X_n = top_mkb.groupby('month_year').sum('PATIENT_ID_COUNT').copy()
# days within a week
X_n["day"] = X_n.index.dayofweek # the x-axis (freq)
X_n["week"] = X_n.index.week # the seasonal period (period)
# days within a year
X_n["month"] = X_n.index.month
X_n["year"] = X_n.index.year
fig, ax0 = plt.subplots(1, 1, figsize=(11, 6))
#seasonal_plot(X_n, y="PATIENT_ID_COUNT", period="week", freq="day", ax=ax0)
#seasonal_plot(X_n, y="PATIENT_ID_COUNT", period="year", freq="month", ax=ax1);
seasonal_plot(X_n, y="PATIENT_ID_COUNT", period="year", freq="month", ax=ax0);
X_n.head()
| PATIENT_ID_COUNT | day | week | month | year | |
|---|---|---|---|---|---|
| month_year | |||||
| 2018-01-01 | 12085 | 0 | 1 | 1 | 2018 |
| 2018-02-01 | 15845 | 3 | 5 | 2 | 2018 |
| 2018-03-01 | 18703 | 3 | 9 | 3 | 2018 |
| 2018-04-01 | 15537 | 6 | 13 | 4 | 2018 |
| 2018-05-01 | 8146 | 1 | 18 | 5 | 2018 |
type(X_n.PATIENT_ID_COUNT.head())
pandas.core.series.Series
plot_periodogram(X_n.PATIENT_ID_COUNT);
pd.DatetimeIndex(X_n.index)
DatetimeIndex(['2018-01-01', '2018-02-01', '2018-03-01', '2018-04-01',
'2018-05-01', '2018-06-01', '2018-07-01', '2018-08-01',
'2018-09-01', '2018-10-01', '2018-11-01', '2018-12-01',
'2019-01-01', '2019-02-01', '2019-03-01', '2019-04-01',
'2019-05-01', '2019-06-01', '2019-07-01', '2019-08-01',
'2019-09-01', '2019-10-01', '2019-11-01', '2019-12-01',
'2020-01-01', '2020-02-01', '2020-03-01', '2020-04-01',
'2020-05-01', '2020-06-01', '2020-07-01', '2020-08-01',
'2020-09-01', '2020-10-01', '2020-11-01', '2020-12-01',
'2021-01-01', '2021-02-01', '2021-03-01', '2021-04-01',
'2021-05-01', '2021-06-01', '2021-07-01', '2021-08-01',
'2021-09-01', '2021-10-01', '2021-11-01', '2021-12-01',
'2022-01-01', '2022-02-01', '2022-03-01'],
dtype='datetime64[ns]', name='month_year', freq=None)
from statsmodels.tsa.deterministic import CalendarFourier, DeterministicProcess
fourier = CalendarFourier(freq="A", order=4) # 10 sin/cos pairs for "A"nnual seasonality
dp_s = DeterministicProcess(
#index=X_n.index,
index=pd.DatetimeIndex(X_n.index),
constant=True, # dummy feature for bias (y-intercept)
order=1, # trend (order 1 means linear)
seasonal=True, # weekly seasonality (indicators)
additional_terms=[fourier], # annual seasonality (fourier)
drop=True, # drop terms to avoid collinearity
)
X_s = dp_s.in_sample() # create features for dates in tunnel.index
X_s
| const | trend | s(2,12) | s(3,12) | s(4,12) | s(5,12) | s(6,12) | s(7,12) | s(8,12) | s(9,12) | ... | s(11,12) | s(12,12) | sin(1,freq=A-DEC) | cos(1,freq=A-DEC) | sin(2,freq=A-DEC) | cos(2,freq=A-DEC) | sin(3,freq=A-DEC) | cos(3,freq=A-DEC) | sin(4,freq=A-DEC) | cos(4,freq=A-DEC) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| month_year | |||||||||||||||||||||
| 2018-01-01 | 1.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.000000 | 1.000000 | 0.000000 | 1.000000 | 0.000000e+00 | 1.000000 | 0.000000 | 1.000000 |
| 2018-02-01 | 1.0 | 2.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.508671 | 0.860961 | 0.875892 | 0.482508 | 9.995463e-01 | -0.030120 | 0.845249 | -0.534373 |
| 2018-03-01 | 1.0 | 3.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.849817 | 0.527078 | 0.895839 | -0.444378 | 9.453675e-02 | -0.995521 | -0.796183 | -0.605056 |
| 2018-04-01 | 1.0 | 4.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.999769 | 0.021516 | 0.043022 | -0.999074 | -9.979172e-01 | -0.064508 | -0.085965 | 0.996298 |
| 2018-05-01 | 1.0 | 5.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.880012 | -0.474951 | -0.835925 | -0.548843 | -8.596480e-02 | 0.996298 | 0.917584 | -0.397543 |
| 2018-06-01 | 1.0 | 6.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.516062 | -0.856551 | -0.884068 | 0.467359 | 9.984354e-01 | 0.055917 | -0.826354 | -0.563151 |
| 2018-07-01 | 1.0 | 7.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.025818 | -0.999667 | -0.051620 | 0.998667 | 7.738648e-02 | -0.997001 | -0.103102 | 0.994671 |
| 2018-08-01 | 1.0 | 8.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | ... | 0.0 | 0.0 | -0.486273 | -0.873807 | 0.849817 | 0.527078 | -9.988797e-01 | -0.047321 | 0.895839 | -0.444378 |
| 2018-09-01 | 1.0 | 9.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | -0.863142 | -0.504961 | 0.871706 | -0.490029 | -1.721336e-02 | 0.999852 | -0.854322 | -0.519744 |
| 2018-10-01 | 1.0 | 10.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | -0.999917 | -0.012910 | 0.025818 | -0.999667 | 9.992500e-01 | 0.038722 | -0.051620 | 0.998667 |
| 2018-11-01 | 1.0 | 11.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 1.0 | 0.0 | -0.867456 | 0.497513 | -0.863142 | -0.504961 | 8.606997e-03 | -0.999963 | 0.871706 | -0.490029 |
| 2018-12-01 | 1.0 | 12.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 1.0 | -0.508671 | 0.860961 | -0.875892 | 0.482508 | -9.995463e-01 | -0.030120 | -0.845249 | -0.534373 |
| 2019-01-01 | 1.0 | 13.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.000000 | 1.000000 | 0.000000 | 1.000000 | 0.000000e+00 | 1.000000 | 0.000000 | 1.000000 |
| 2019-02-01 | 1.0 | 14.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.508671 | 0.860961 | 0.875892 | 0.482508 | 9.995463e-01 | -0.030120 | 0.845249 | -0.534373 |
| 2019-03-01 | 1.0 | 15.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.849817 | 0.527078 | 0.895839 | -0.444378 | 9.453675e-02 | -0.995521 | -0.796183 | -0.605056 |
| 2019-04-01 | 1.0 | 16.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.999769 | 0.021516 | 0.043022 | -0.999074 | -9.979172e-01 | -0.064508 | -0.085965 | 0.996298 |
| 2019-05-01 | 1.0 | 17.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.880012 | -0.474951 | -0.835925 | -0.548843 | -8.596480e-02 | 0.996298 | 0.917584 | -0.397543 |
| 2019-06-01 | 1.0 | 18.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.516062 | -0.856551 | -0.884068 | 0.467359 | 9.984354e-01 | 0.055917 | -0.826354 | -0.563151 |
| 2019-07-01 | 1.0 | 19.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.025818 | -0.999667 | -0.051620 | 0.998667 | 7.738648e-02 | -0.997001 | -0.103102 | 0.994671 |
| 2019-08-01 | 1.0 | 20.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | ... | 0.0 | 0.0 | -0.486273 | -0.873807 | 0.849817 | 0.527078 | -9.988797e-01 | -0.047321 | 0.895839 | -0.444378 |
| 2019-09-01 | 1.0 | 21.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | -0.863142 | -0.504961 | 0.871706 | -0.490029 | -1.721336e-02 | 0.999852 | -0.854322 | -0.519744 |
| 2019-10-01 | 1.0 | 22.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | -0.999917 | -0.012910 | 0.025818 | -0.999667 | 9.992500e-01 | 0.038722 | -0.051620 | 0.998667 |
| 2019-11-01 | 1.0 | 23.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 1.0 | 0.0 | -0.867456 | 0.497513 | -0.863142 | -0.504961 | 8.606997e-03 | -0.999963 | 0.871706 | -0.490029 |
| 2019-12-01 | 1.0 | 24.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 1.0 | -0.508671 | 0.860961 | -0.875892 | 0.482508 | -9.995463e-01 | -0.030120 | -0.845249 | -0.534373 |
| 2020-01-01 | 1.0 | 25.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.000000 | 1.000000 | 0.000000 | 1.000000 | 0.000000e+00 | 1.000000 | 0.000000 | 1.000000 |
| 2020-02-01 | 1.0 | 26.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.507415 | 0.861702 | 0.874481 | 0.485060 | 9.996685e-01 | -0.025748 | 0.848351 | -0.529434 |
| 2020-03-01 | 1.0 | 27.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.857315 | 0.514793 | 0.882679 | -0.469977 | 5.147875e-02 | -0.998674 | -0.829677 | -0.558244 |
| 2020-04-01 | 1.0 | 28.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.999963 | 0.008583 | 0.017166 | -0.999853 | -9.996685e-01 | -0.025748 | -0.034328 | 0.999411 |
| 2020-05-01 | 1.0 | 29.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.874481 | -0.485060 | -0.848351 | -0.529434 | -5.147875e-02 | 0.998674 | 0.898292 | -0.439400 |
| 2020-06-01 | 1.0 | 30.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.507415 | -0.861702 | -0.874481 | 0.485060 | 9.996685e-01 | 0.025748 | -0.848351 | -0.529434 |
| 2020-07-01 | 1.0 | 31.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.017166 | -0.999853 | -0.034328 | 0.999411 | 5.147875e-02 | -0.998674 | -0.068615 | 0.997643 |
| 2020-08-01 | 1.0 | 32.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | ... | 0.0 | 0.0 | -0.492548 | -0.870285 | 0.857315 | 0.514793 | -9.996685e-01 | -0.025748 | 0.882679 | -0.469977 |
| 2020-09-01 | 1.0 | 33.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | -0.866025 | -0.500000 | 0.866025 | -0.500000 | -4.898587e-16 | 1.000000 | -0.866025 | -0.500000 |
| 2020-10-01 | 1.0 | 34.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | -0.999963 | -0.008583 | 0.017166 | -0.999853 | 9.996685e-01 | 0.025748 | -0.034328 | 0.999411 |
| 2020-11-01 | 1.0 | 35.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 1.0 | 0.0 | -0.866025 | 0.500000 | -0.866025 | -0.500000 | 6.123234e-16 | -1.000000 | 0.866025 | -0.500000 |
| 2020-12-01 | 1.0 | 36.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 1.0 | -0.507415 | 0.861702 | -0.874481 | 0.485060 | -9.996685e-01 | -0.025748 | -0.848351 | -0.529434 |
| 2021-01-01 | 1.0 | 37.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.000000 | 1.000000 | 0.000000 | 1.000000 | 0.000000e+00 | 1.000000 | 0.000000 | 1.000000 |
| 2021-02-01 | 1.0 | 38.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.508671 | 0.860961 | 0.875892 | 0.482508 | 9.995463e-01 | -0.030120 | 0.845249 | -0.534373 |
| 2021-03-01 | 1.0 | 39.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.849817 | 0.527078 | 0.895839 | -0.444378 | 9.453675e-02 | -0.995521 | -0.796183 | -0.605056 |
| 2021-04-01 | 1.0 | 40.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.999769 | 0.021516 | 0.043022 | -0.999074 | -9.979172e-01 | -0.064508 | -0.085965 | 0.996298 |
| 2021-05-01 | 1.0 | 41.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.880012 | -0.474951 | -0.835925 | -0.548843 | -8.596480e-02 | 0.996298 | 0.917584 | -0.397543 |
| 2021-06-01 | 1.0 | 42.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.516062 | -0.856551 | -0.884068 | 0.467359 | 9.984354e-01 | 0.055917 | -0.826354 | -0.563151 |
| 2021-07-01 | 1.0 | 43.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.025818 | -0.999667 | -0.051620 | 0.998667 | 7.738648e-02 | -0.997001 | -0.103102 | 0.994671 |
| 2021-08-01 | 1.0 | 44.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | ... | 0.0 | 0.0 | -0.486273 | -0.873807 | 0.849817 | 0.527078 | -9.988797e-01 | -0.047321 | 0.895839 | -0.444378 |
| 2021-09-01 | 1.0 | 45.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | ... | 0.0 | 0.0 | -0.863142 | -0.504961 | 0.871706 | -0.490029 | -1.721336e-02 | 0.999852 | -0.854322 | -0.519744 |
| 2021-10-01 | 1.0 | 46.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | -0.999917 | -0.012910 | 0.025818 | -0.999667 | 9.992500e-01 | 0.038722 | -0.051620 | 0.998667 |
| 2021-11-01 | 1.0 | 47.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 1.0 | 0.0 | -0.867456 | 0.497513 | -0.863142 | -0.504961 | 8.606997e-03 | -0.999963 | 0.871706 | -0.490029 |
| 2021-12-01 | 1.0 | 48.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 1.0 | -0.508671 | 0.860961 | -0.875892 | 0.482508 | -9.995463e-01 | -0.030120 | -0.845249 | -0.534373 |
| 2022-01-01 | 1.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.000000 | 1.000000 | 0.000000 | 1.000000 | 0.000000e+00 | 1.000000 | 0.000000 | 1.000000 |
| 2022-02-01 | 1.0 | 50.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.508671 | 0.860961 | 0.875892 | 0.482508 | 9.995463e-01 | -0.030120 | 0.845249 | -0.534373 |
| 2022-03-01 | 1.0 | 51.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | ... | 0.0 | 0.0 | 0.849817 | 0.527078 | 0.895839 | -0.444378 | 9.453675e-02 | -0.995521 | -0.796183 | -0.605056 |
51 rows × 21 columns
dp_s.out_of_sample(steps=51,forecast_index=X_n.index)
X_n.index+51*np.timedelta64(1, 'M')
DatetimeIndex(['2022-04-02 06:44:06', '2022-05-03 06:44:06',
'2022-05-31 06:44:06', '2022-07-01 06:44:06',
'2022-07-31 06:44:06', '2022-08-31 06:44:06',
'2022-09-30 06:44:06', '2022-10-31 06:44:06',
'2022-12-01 06:44:06', '2022-12-31 06:44:06',
'2023-01-31 06:44:06', '2023-03-02 06:44:06',
'2023-04-02 06:44:06', '2023-05-03 06:44:06',
'2023-05-31 06:44:06', '2023-07-01 06:44:06',
'2023-07-31 06:44:06', '2023-08-31 06:44:06',
'2023-09-30 06:44:06', '2023-10-31 06:44:06',
'2023-12-01 06:44:06', '2023-12-31 06:44:06',
'2024-01-31 06:44:06', '2024-03-01 06:44:06',
'2024-04-01 06:44:06', '2024-05-02 06:44:06',
'2024-05-31 06:44:06', '2024-07-01 06:44:06',
'2024-07-31 06:44:06', '2024-08-31 06:44:06',
'2024-09-30 06:44:06', '2024-10-31 06:44:06',
'2024-12-01 06:44:06', '2024-12-31 06:44:06',
'2025-01-31 06:44:06', '2025-03-02 06:44:06',
'2025-04-02 06:44:06', '2025-05-03 06:44:06',
'2025-05-31 06:44:06', '2025-07-01 06:44:06',
'2025-07-31 06:44:06', '2025-08-31 06:44:06',
'2025-09-30 06:44:06', '2025-10-31 06:44:06',
'2025-12-01 06:44:06', '2025-12-31 06:44:06',
'2026-01-31 06:44:06', '2026-03-02 06:44:06',
'2026-04-02 06:44:06', '2026-05-03 06:44:06',
'2026-05-31 06:44:06'],
dtype='datetime64[ns]', name='month_year', freq=None)
pd.date_range(start='2022-04-01', end='2026-07-01', freq='M').shape
(51,)
X_n['PATIENT_ID_COUNT'].tail()
month_year 2021-11-01 13441 2021-12-01 27815 2022-01-01 18372 2022-02-01 20982 2022-03-01 8589 Name: PATIENT_ID_COUNT, dtype: int32
X_n.index[-1]
Timestamp('2022-03-01 00:00:00')
y_s = X_n['PATIENT_ID_COUNT']
model = LinearRegression(fit_intercept=False)
_ = model.fit(X_s, y_s)
y_pred = pd.Series(model.predict(X_s), index=y_s.index)
#X_fore = dp_s.out_of_sample(steps=51, forecast_index=X_n.index+51*np.timedelta64(1, 'M'))
#X_fore = dp_s.out_of_sample(steps=51, forecast_index=pd.date_range(start='2022-04-01', end='2026-07-01', freq='M'))
#y_fore = pd.Series(model.predict(X_fore), index=X_fore.index)
ax = y_s.plot(color='0.25', style='.', title="TOP MKB - Seasonal Forecast")
ax = y_pred.plot(ax=ax, label="Seasonal")
#ax = y_fore.plot(ax=ax, label="Seasonal Forecast", color='C3')
#ax.set_xlim(X_n.index[0], X_n.index[-1])
_ = ax.legend()
# Correlations https://www.kaggle.com/code/ryanholbrook/time-series-as-features/tutorial
from pathlib import Path
from warnings import simplefilter
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from scipy.signal import periodogram
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from statsmodels.graphics.tsaplots import plot_pacf
simplefilter("ignore")
# Set Matplotlib defaults
plt.style.use("seaborn-whitegrid")
plt.rc("figure", autolayout=True, figsize=(11, 4))
plt.rc(
"axes",
labelweight="bold",
labelsize="large",
titleweight="bold",
titlesize=16,
titlepad=10,
)
plot_params = dict(
color="0.75",
style=".-",
markeredgecolor="0.25",
markerfacecolor="0.25",
)
%config InlineBackend.figure_format = 'retina'
def lagplot(x, y=None, lag=1, standardize=False, ax=None, **kwargs):
from matplotlib.offsetbox import AnchoredText
x_ = x.shift(lag)
if standardize:
x_ = (x_ - x_.mean()) / x_.std()
if y is not None:
y_ = (y - y.mean()) / y.std() if standardize else y
else:
y_ = x
corr = y_.corr(x_)
if ax is None:
fig, ax = plt.subplots()
scatter_kws = dict(
alpha=0.75,
s=3,
)
line_kws = dict(color='C3', )
ax = sns.regplot(x=x_,
y=y_,
scatter_kws=scatter_kws,
line_kws=line_kws,
lowess=True,
ax=ax,
**kwargs)
at = AnchoredText(
f"{corr:.2f}",
prop=dict(size="large"),
frameon=True,
loc="upper left",
)
at.patch.set_boxstyle("square, pad=0.0")
ax.add_artist(at)
ax.set(title=f"Lag {lag}", xlabel=x_.name, ylabel=y_.name)
return ax
def plot_lags(x, y=None, lags=6, nrows=1, lagplot_kwargs={}, **kwargs):
import math
kwargs.setdefault('nrows', nrows)
kwargs.setdefault('ncols', math.ceil(lags / nrows))
kwargs.setdefault('figsize', (kwargs['ncols'] * 2, nrows * 2 + 0.5))
fig, axs = plt.subplots(sharex=True, sharey=True, squeeze=False, **kwargs)
for ax, k in zip(fig.get_axes(), range(kwargs['nrows'] * kwargs['ncols'])):
if k + 1 <= lags:
ax = lagplot(x, y, lag=k + 1, ax=ax, **lagplot_kwargs)
ax.set_title(f"Lag {k + 1}", fontdict=dict(fontsize=14))
ax.set(xlabel="", ylabel="")
else:
ax.axis('off')
plt.setp(axs[-1, :], xlabel=x.name)
plt.setp(axs[:, 0], ylabel=y.name if y is not None else x.name)
fig.tight_layout(w_pad=0.1, h_pad=0.1)
return fig
#data_dir = Path("../input/ts-course-data")
#flu_trends = pd.read_csv(data_dir / "flu-trends.csv")
#flu_trends.set_index(
# pd.PeriodIndex(flu_trends.Week, freq="W"),
# inplace=True,
#)
#flu_trends.drop("Week", axis=1, inplace=True)
#ax = flu_trends.FluVisits.plot(title='Flu Trends', **plot_params)
#_ = ax.set(ylabel="Office Visits")
flu_trends = X_n.copy()
flu_trends.head()
| PATIENT_ID_COUNT | day | week | month | year | |
|---|---|---|---|---|---|
| month_year | |||||
| 2018-01-01 | 12085 | 0 | 1 | 1 | 2018 |
| 2018-02-01 | 15845 | 3 | 5 | 2 | 2018 |
| 2018-03-01 | 18703 | 3 | 9 | 3 | 2018 |
| 2018-04-01 | 15537 | 6 | 13 | 4 | 2018 |
| 2018-05-01 | 8146 | 1 | 18 | 5 | 2018 |
flu_trends.
Index(['PATIENT_ID_COUNT', 'day', 'week', 'month', 'year'], dtype='object')
flu_trends.set_index(pd.PeriodIndex(flu_trends.index, freq="M"), inplace=True)
ax = flu_trends.PATIENT_ID_COUNT.plot(title='Flu Trends', **plot_params)
_ = ax.set(ylabel="Office Visits")
# Import Meteostat library and dependencies
from datetime import datetime
import matplotlib.pyplot as plt
from meteostat import Point, Daily, Monthly
# Set time period
start = datetime(2018, 1, 1)
end = datetime(2022, 4, 30)
# Create Point for Kaliningrad
location = Point(54.710128, 20.5105838, 0)
# Get daily data for 2018
data = Monthly(location, start, end)
data = data.fetch()
# Plot line chart including average, minimum and maximum temperature
data.plot(y=['tavg', 'tmin', 'tmax'], marker='o')
plt.show()
flu_trends.shape
(51, 5)
data
| tavg | tmin | tmax | prcp | wspd | pres | tsun | |
|---|---|---|---|---|---|---|---|
| time | |||||||
| 2018-01-01 | 0.1 | -2.1 | 2.2 | 79.4 | 13.0 | NaN | NaN |
| 2018-02-01 | -4.3 | -6.9 | -0.7 | 22.1 | NaN | NaN | NaN |
| 2018-03-01 | -0.8 | -3.5 | 3.0 | 20.6 | NaN | NaN | NaN |
| 2018-04-01 | 10.3 | 5.4 | 17.2 | 43.7 | NaN | NaN | NaN |
| 2018-05-01 | 15.9 | 10.2 | 22.9 | 35.5 | 10.5 | NaN | NaN |
| 2018-06-01 | 17.2 | 12.2 | 23.2 | 34.4 | NaN | NaN | NaN |
| 2018-07-01 | 20.3 | 16.5 | 25.5 | 94.2 | NaN | 1012.7 | NaN |
| 2018-08-01 | 19.6 | 14.7 | 25.4 | 65.9 | 10.0 | 1015.7 | NaN |
| 2018-09-01 | 15.3 | 10.4 | 21.3 | 40.0 | 10.0 | 1017.8 | NaN |
| 2018-10-01 | 10.3 | 5.1 | 15.4 | 61.6 | 12.9 | 1015.8 | NaN |
| 2018-11-01 | 4.5 | 2.1 | 6.9 | 29.5 | 11.1 | 1024.6 | NaN |
| 2018-12-01 | 1.9 | -0.2 | 3.3 | 86.5 | 12.0 | 1014.7 | NaN |
| 2019-01-01 | -1.4 | -4.4 | 0.9 | 87.4 | 14.6 | 1007.5 | NaN |
| 2019-02-01 | 3.0 | 0.6 | 5.6 | 37.2 | 15.3 | 1017.8 | NaN |
| 2019-03-01 | 4.7 | 1.3 | 8.9 | 56.5 | 16.6 | 1009.8 | NaN |
| 2019-04-01 | 8.4 | 3.9 | 15.4 | 3.5 | 13.2 | 1020.8 | NaN |
| 2019-05-01 | 11.9 | 7.9 | 17.3 | 56.9 | 13.0 | 1012.5 | NaN |
| 2019-06-01 | 20.1 | 14.3 | 27.2 | 65.7 | 11.1 | 1017.5 | NaN |
| 2019-07-01 | 17.6 | 13.3 | 22.3 | 79.0 | 13.2 | 1011.6 | NaN |
| 2019-08-01 | 18.6 | 13.1 | 24.8 | NaN | 9.2 | 1016.4 | NaN |
| 2019-09-01 | 14.1 | 9.5 | 18.7 | NaN | 12.8 | 1014.5 | NaN |
| 2019-10-01 | 10.7 | 6.9 | 14.2 | NaN | 11.5 | 1012.6 | NaN |
| 2019-11-01 | 6.0 | 3.0 | 8.1 | NaN | 12.9 | 1009.5 | NaN |
| 2019-12-01 | 4.1 | 1.5 | 5.9 | 55.2 | 14.4 | 1009.5 | NaN |
| 2020-01-01 | 4.5 | 1.8 | 6.0 | NaN | 16.4 | 1016.5 | NaN |
| 2020-02-01 | 4.2 | 1.1 | 7.3 | 79.5 | 17.4 | 1005.7 | NaN |
| 2020-03-01 | 4.6 | 0.6 | 9.0 | 47.5 | 13.9 | 1016.9 | NaN |
| 2020-04-01 | 7.5 | 1.7 | 13.8 | 8.7 | 15.3 | 1016.4 | NaN |
| 2020-05-01 | 10.4 | 5.6 | 15.9 | 86.7 | 14.6 | 1016.2 | NaN |
| 2020-06-01 | 17.9 | 13.0 | 23.6 | 93.2 | 10.7 | 1012.5 | NaN |
| 2020-07-01 | 17.4 | 12.0 | 22.9 | NaN | 13.4 | 1013.5 | NaN |
| 2020-08-01 | 19.0 | 13.5 | 23.8 | NaN | 10.8 | 1013.8 | NaN |
| 2020-09-01 | 15.7 | 11.3 | 20.3 | NaN | 14.6 | 1015.3 | NaN |
| 2020-10-01 | 11.1 | 7.9 | 14.0 | NaN | 14.3 | 1011.5 | NaN |
| 2020-11-01 | 7.2 | 4.8 | 9.2 | NaN | 16.5 | 1020.6 | NaN |
| 2020-12-01 | 2.2 | 0.3 | 3.8 | NaN | 17.3 | 1013.2 | NaN |
| 2021-01-01 | -1.3 | -3.5 | 0.7 | NaN | 14.0 | 1008.1 | NaN |
| 2021-02-01 | -3.0 | -7.0 | 0.7 | NaN | 13.9 | 1020.2 | NaN |
| 2021-03-01 | 3.3 | -0.2 | 6.5 | NaN | 17.8 | 1016.2 | NaN |
| 2021-04-01 | 6.0 | 1.2 | 10.2 | NaN | 17.5 | 1014.0 | NaN |
| 2021-05-01 | 11.6 | 6.6 | 15.9 | NaN | 15.7 | 1010.0 | NaN |
| 2021-06-01 | 18.7 | 12.6 | 23.3 | NaN | 12.0 | 1016.9 | NaN |
| 2021-07-01 | 21.7 | 16.5 | 26.2 | NaN | 11.8 | 1013.0 | NaN |
| 2021-08-01 | 17.0 | 13.2 | 20.8 | NaN | 13.8 | 1011.9 | NaN |
| 2021-09-01 | 13.5 | 9.6 | 17.3 | NaN | 14.5 | 1017.5 | NaN |
| 2021-10-01 | 9.5 | 5.4 | 13.7 | NaN | 15.6 | 1017.2 | NaN |
| 2021-11-01 | 5.8 | 3.5 | 7.8 | NaN | 17.7 | 1011.3 | NaN |
| 2021-12-01 | -0.8 | -3.1 | 1.3 | NaN | 17.4 | 1011.9 | NaN |
| 2022-01-01 | 2.2 | -0.1 | 4.1 | NaN | 23.8 | 1011.7 | NaN |
| 2022-02-01 | 3.3 | 0.8 | 5.5 | NaN | 21.7 | 1007.4 | NaN |
| 2022-03-01 | 2.3 | -2.5 | 7.5 | NaN | 13.4 | 1027.5 | NaN |
| 2022-04-01 | 6.6 | 2.4 | 10.5 | NaN | 18.1 | 1012.3 | NaN |
data.shape
(52, 7)
# Correlation of flu trends with temperature weather
flu_trends.PATIENT_ID_COUNT.pct_change().plot()
<AxesSubplot:xlabel='month_year'>
flu_trends.PATIENT_ID_COUNT.pct_change()[1:].corr(data.tavg[:-1].pct_change()[1:])
nan
for sh in range(12):
cor1 = pd.Series(flu_trends.PATIENT_ID_COUNT.pct_change()[1:]).reset_index(drop=True).shift(-sh).fillna(0)
cor2 = pd.Series(data.tavg[:-1].pct_change()[1:]).reset_index(drop=True)
print(cor1.corr(cor2))
-0.06417580979741487 0.02250092114205408 0.14118144584977046 0.22176692198809464 0.12883386102187286 0.019168390533101202 -0.060192140185869866 -0.1621198229272845 -0.03493780901316206 0.03750022489283038 -0.04497970898513469 0.02043814618039486
for sh in range(12):
cor1 = pd.Series(flu_trends.PATIENT_ID_COUNT.diff()[1:]).reset_index(drop=True).shift(sh).fillna(0)
cor2 = pd.Series(data.tavg[:-1].diff()[1:]).reset_index(drop=True)
print(cor1.corr(cor2))
-0.2941965409111408 -0.16278038156263508 -0.3144270207615394 -0.1985883753370914 0.19419555431362068 0.08239732298087364 0.2955482434617382 0.2543930268057727 0.28567833315018476 0.10404052903111528 -0.07941739892006597 -0.10885793969789104
for sh in range(12):
cor = pd.Series(data.tavg).reset_index(drop=True).corr(pd.Series(flu_trends.PATIENT_ID_COUNT).reset_index(drop=True).shift(-sh).fillna(0))
print(cor)
-0.538324561245258 -0.2678961701573419 0.06754337664020182 0.4180993072467921 0.6149283850857821 0.6434223700554677 0.5356218527096469 0.279249588312017 0.0004965990130966684 -0.21641064847307917 -0.38776846208139465 -0.4382146984850754
pd.Series(data.tavg).reset_index(drop=True).diff().plot(marker='o')
<AxesSubplot:>
ft_moving_average = flu_trends.PATIENT_ID_COUNT.rolling(
window=8, # 365-day window
center=True, # puts the average at the center of the window
min_periods=4, # choose about half the window size
).mean() # compute the mean (could also do median, std, min, max, ...)
ft_moving_average.plot(marker='o')
<AxesSubplot:xlabel='month_year'>
data.tavg[:-1].pct_change()[1:]
time 2018-02-01 -44.000000 2018-03-01 -0.813953 2018-04-01 -13.875000 2018-05-01 0.543689 2018-06-01 0.081761 2018-07-01 0.180233 2018-08-01 -0.034483 2018-09-01 -0.219388 2018-10-01 -0.326797 2018-11-01 -0.563107 2018-12-01 -0.577778 2019-01-01 -1.736842 2019-02-01 -3.142857 2019-03-01 0.566667 2019-04-01 0.787234 2019-05-01 0.416667 2019-06-01 0.689076 2019-07-01 -0.124378 2019-08-01 0.056818 2019-09-01 -0.241935 2019-10-01 -0.241135 2019-11-01 -0.439252 2019-12-01 -0.316667 2020-01-01 0.097561 2020-02-01 -0.066667 2020-03-01 0.095238 2020-04-01 0.630435 2020-05-01 0.386667 2020-06-01 0.721154 2020-07-01 -0.027933 2020-08-01 0.091954 2020-09-01 -0.173684 2020-10-01 -0.292994 2020-11-01 -0.351351 2020-12-01 -0.694444 2021-01-01 -1.590909 2021-02-01 1.307692 2021-03-01 -2.100000 2021-04-01 0.818182 2021-05-01 0.933333 2021-06-01 0.612069 2021-07-01 0.160428 2021-08-01 -0.216590 2021-09-01 -0.205882 2021-10-01 -0.296296 2021-11-01 -0.389474 2021-12-01 -1.137931 2022-01-01 -3.750000 2022-02-01 0.500000 2022-03-01 -0.303030 Freq: MS, Name: tavg, dtype: float64
flu_trends.PATIENT_ID_COUNT.pct_change()
month_year 2018-01 NaN 2018-02 0.311129 2018-03 0.180372 2018-04 -0.169278 2018-05 -0.475703 2018-06 -0.272895 2018-07 -0.045754 2018-08 0.279547 2018-09 0.641869 2018-10 0.174162 2018-11 -0.125161 2018-12 0.040748 2019-01 0.057823 2019-02 0.684987 2019-03 -0.372757 2019-04 -0.225056 2019-05 -0.391071 2019-06 -0.176348 2019-07 0.071429 2019-08 0.054146 2019-09 1.029374 2019-10 0.053706 2019-11 -0.264375 2019-12 0.182449 2020-01 -0.100794 2020-02 0.658253 2020-03 -0.033914 2020-04 -0.671359 2020-05 -0.564710 2020-06 0.036013 2020-07 1.300323 2020-08 -0.042172 2020-09 1.073381 2020-10 0.407362 2020-11 0.386395 2020-12 0.038727 2021-01 -0.574522 2021-02 0.139031 2021-03 -0.019007 2021-04 -0.024977 2021-05 -0.192315 2021-06 0.018424 2021-07 0.149531 2021-08 0.103486 2021-09 0.899540 2021-10 0.107627 2021-11 -0.423381 2021-12 1.069414 2022-01 -0.339493 2022-02 0.142064 2022-03 -0.590649 Freq: M, Name: PATIENT_ID_COUNT, dtype: float64
pd.Series(list(range(10))).corr(pd.Series(list(range(10))))
0.9999999999999999
pd.Series(data.tavg[:-1].pct_change())[1:]
time 2018-02-01 -44.000000 2018-03-01 -0.813953 2018-04-01 -13.875000 2018-05-01 0.543689 2018-06-01 0.081761 2018-07-01 0.180233 2018-08-01 -0.034483 2018-09-01 -0.219388 2018-10-01 -0.326797 2018-11-01 -0.563107 2018-12-01 -0.577778 2019-01-01 -1.736842 2019-02-01 -3.142857 2019-03-01 0.566667 2019-04-01 0.787234 2019-05-01 0.416667 2019-06-01 0.689076 2019-07-01 -0.124378 2019-08-01 0.056818 2019-09-01 -0.241935 2019-10-01 -0.241135 2019-11-01 -0.439252 2019-12-01 -0.316667 2020-01-01 0.097561 2020-02-01 -0.066667 2020-03-01 0.095238 2020-04-01 0.630435 2020-05-01 0.386667 2020-06-01 0.721154 2020-07-01 -0.027933 2020-08-01 0.091954 2020-09-01 -0.173684 2020-10-01 -0.292994 2020-11-01 -0.351351 2020-12-01 -0.694444 2021-01-01 -1.590909 2021-02-01 1.307692 2021-03-01 -2.100000 2021-04-01 0.818182 2021-05-01 0.933333 2021-06-01 0.612069 2021-07-01 0.160428 2021-08-01 -0.216590 2021-09-01 -0.205882 2021-10-01 -0.296296 2021-11-01 -0.389474 2021-12-01 -1.137931 2022-01-01 -3.750000 2022-02-01 0.500000 2022-03-01 -0.303030 Freq: MS, Name: tavg, dtype: float64
data.tavg[:-1].pct_change().plot()
<AxesSubplot:xlabel='time'>
plt.scatter(flu_trends.PATIENT_ID_COUNT, data.tavg[:-1])
<matplotlib.collections.PathCollection at 0x2043f915b50>
_ = plot_lags(flu_trends.PATIENT_ID_COUNT, lags=12, nrows=2)
_ = plot_pacf(flu_trends.PATIENT_ID_COUNT, lags=12)
_ = plot_lags(flu_trends.PATIENT_ID_COUNT[-36:], lags=12, nrows=2)
_ = plot_pacf(flu_trends.PATIENT_ID_COUNT[-36:], lags=12)
#ACF
from statsmodels.graphics.tsaplots import plot_acf
plot_acf(flu_trends.PATIENT_ID_COUNT, lags=12)
# https://medium.com/@krzysztofdrelczuk/acf-autocorrelation-function-simple-explanation-with-python-example-492484c32711
# https://github.com/kdrelczuk/medium
from pandas import read_csv
from statsmodels.graphics.tsaplots import plot_acf
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
data = pd.read_csv('monthly-sunspots.txt').drop(['Month'],axis=1)#.head(100)
data_a = data.to_numpy().T[0]
data_a
plt.figure(figsize=(20,10))
plt.plot(data_a)
plt.rc("figure", figsize=(20,10))
plt.figure(figsize=(20,10))
plot_acf(data_a, lags=90)
plt.show()
data = pd.read_csv('AirPassengers.csv').drop(['Month'],axis=1)#.head(100)
data_a = data.to_numpy().T[0]
data_a
plt.figure(figsize=(20,10))
plt.plot(data_a)
plt.rc("figure", figsize=(20,10))
plt.figure(figsize=(20,10))
plot_acf(data_a, lags=50)
plt.show()
<Figure size 1440x720 with 0 Axes>
<Figure size 1440x720 with 0 Axes>
_ = plot_lags(data.tavg, lags=12, nrows=2)
_ = plot_pacf(data.tavg, lags=12)
data.tavg
time 2018-01-01 0.1 2018-02-01 -4.3 2018-03-01 -0.8 2018-04-01 10.3 2018-05-01 15.9 2018-06-01 17.2 2018-07-01 20.3 2018-08-01 19.6 2018-09-01 15.3 2018-10-01 10.3 2018-11-01 4.5 2018-12-01 1.9 2019-01-01 -1.4 2019-02-01 3.0 2019-03-01 4.7 2019-04-01 8.4 2019-05-01 11.9 2019-06-01 20.1 2019-07-01 17.6 2019-08-01 18.6 2019-09-01 14.1 2019-10-01 10.7 2019-11-01 6.0 2019-12-01 4.1 2020-01-01 4.5 2020-02-01 4.2 2020-03-01 4.6 2020-04-01 7.5 2020-05-01 10.4 2020-06-01 17.9 2020-07-01 17.4 2020-08-01 19.0 2020-09-01 15.7 2020-10-01 11.1 2020-11-01 7.2 2020-12-01 2.2 2021-01-01 -1.3 2021-02-01 -3.0 2021-03-01 3.3 2021-04-01 6.0 2021-05-01 11.6 2021-06-01 18.7 2021-07-01 21.7 2021-08-01 17.0 2021-09-01 13.5 2021-10-01 9.5 2021-11-01 5.8 2021-12-01 -0.8 2022-01-01 2.2 2022-02-01 3.3 2022-03-01 2.3 2022-04-01 6.6 Freq: MS, Name: tavg, dtype: float64
def make_lags(ts, lags):
return pd.concat(
{
f'y_lag_{i}': ts.shift(i)
for i in range(1, lags + 1)
},
axis=1)
X = make_lags(flu_trends.PATIENT_ID_COUNT, lags=1)
X = X.fillna(0.0)
# Create target series and data splits
y = flu_trends.PATIENT_ID_COUNT.copy()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=10, shuffle=False)
# Fit and predict
model = LinearRegression() # `fit_intercept=True` since we didn't use DeterministicProcess
model.fit(X_train, y_train)
y_pred = pd.Series(model.predict(X_train), index=y_train.index)
y_fore = pd.Series(model.predict(X_test), index=y_test.index)
ax = y_train.plot(**plot_params)
ax = y_test.plot(**plot_params)
ax = y_pred.plot(ax=ax)
_ = y_fore.plot(ax=ax, color='C3')
ax = y_test.plot(**plot_params)
_ = y_fore.plot(ax=ax, color='C3')
# Import Meteostat library and dependencies
from datetime import datetime
import matplotlib.pyplot as plt
from meteostat import Point, Daily
# Set time period
start = datetime(2018, 1, 1)
end = datetime(2022, 4, 30)
# Create Point for Kaliningrad
location = Point(54.710128, 20.5105838, 0)
# Get daily data for 2018
data = Daily(location, start, end)
data = data.fetch()
# Plot line chart including average, minimum and maximum temperature
data.plot(y=['tavg', 'tmin', 'tmax'])
plt.show()
# Import Meteostat library and dependencies
from datetime import datetime
import matplotlib.pyplot as plt
from meteostat import Stations, Monthly
# Set time period
start = datetime(2000, 1, 1)
end = datetime(2018, 12, 31)
# Get Monthly data
data = Monthly('10637', start, end)
data = data.fetch()
# Plot line chart including average, minimum and maximum temperature
data.plot(y=['tavg', 'tmin', 'tmax'])
plt.show()
_ = plot_lags(flu_trends.FluVisits, lags=12, nrows=2)
_ = plot_pacf(flu_trends.FluVisits, lags=12)
train.groupby('MKB_CODE').sum('PATIENT_ID_COUNT').describe()
| PATIENT_ID_COUNT | |
|---|---|
| count | 7644.000000 |
| mean | 1692.536630 |
| std | 17989.725347 |
| min | 1.000000 |
| 25% | 6.000000 |
| 50% | 40.000000 |
| 75% | 284.000000 |
| max | 919789.000000 |
train.groupby('VISIT_MONTH_YEAR').sum('PATIENT_ID_COUNT').describe()
| PATIENT_ID_COUNT | |
|---|---|
| count | 51.000000 |
| mean | 253681.372549 |
| std | 51270.812564 |
| min | 105810.000000 |
| 25% | 234124.000000 |
| 50% | 258120.000000 |
| 75% | 281405.000000 |
| max | 347865.000000 |
grp_by_month = train
grp_by_month['month'] = pd.to_datetime(grp_by_month['VISIT_MONTH_YEAR'], format='%m.%y', errors='ignore')
grp_by_month = grp_by_month.groupby('month').sum('PATIENT_ID_COUNT')
grp_by_month.describe()
| PATIENT_ID_COUNT | |
|---|---|
| count | 51.000000 |
| mean | 253681.372549 |
| std | 51270.812564 |
| min | 105810.000000 |
| 25% | 234124.000000 |
| 50% | 258120.000000 |
| 75% | 281405.000000 |
| max | 347865.000000 |
grp_by_month
| PATIENT_ID_COUNT | |
|---|---|
| month | |
| 2018-01-01 | 221854 |
| 2018-02-01 | 255036 |
| 2018-03-01 | 263285 |
| 2018-04-01 | 258038 |
| 2018-05-01 | 237447 |
| 2018-06-01 | 238109 |
| 2018-07-01 | 239620 |
| 2018-08-01 | 242724 |
| 2018-09-01 | 253018 |
| 2018-10-01 | 287364 |
| 2018-11-01 | 272947 |
| 2018-12-01 | 259683 |
| 2019-01-01 | 240650 |
| 2019-02-01 | 279467 |
| 2019-03-01 | 275218 |
| 2019-04-01 | 290365 |
| 2019-05-01 | 243530 |
| 2019-06-01 | 240687 |
| 2019-07-01 | 262111 |
| 2019-08-01 | 258120 |
| 2019-09-01 | 280559 |
| 2019-10-01 | 304796 |
| 2019-11-01 | 269928 |
| 2019-12-01 | 282251 |
| 2020-01-01 | 189966 |
| 2020-02-01 | 241591 |
| 2020-03-01 | 233576 |
| 2020-04-01 | 111634 |
| 2020-05-01 | 105810 |
| 2020-06-01 | 173373 |
| 2020-07-01 | 217798 |
| 2020-08-01 | 225479 |
| 2020-09-01 | 261412 |
| 2020-10-01 | 263960 |
| 2020-11-01 | 224981 |
| 2020-12-01 | 234672 |
| 2021-01-01 | 150000 |
| 2021-02-01 | 223190 |
| 2021-03-01 | 275785 |
| 2021-04-01 | 313055 |
| 2021-05-01 | 294018 |
| 2021-06-01 | 311045 |
| 2021-07-01 | 316501 |
| 2021-08-01 | 313895 |
| 2021-09-01 | 328436 |
| 2021-10-01 | 347865 |
| 2021-11-01 | 328531 |
| 2021-12-01 | 345437 |
| 2022-01-01 | 190701 |
| 2022-02-01 | 196348 |
| 2022-03-01 | 261884 |
plt.plot(grp_by_month['PATIENT_ID_COUNT'], marker = 'o')
plt.rcParams["figure.figsize"] = (20,30)
plt.show()
!pip install geopy
Requirement already satisfied: geopy in d:\programdata\anaconda3\lib\site-packages (2.2.0)
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Requirement already satisfied: geographiclib<2,>=1.49 in d:\programdata\anaconda3\lib\site-packages (from geopy) (1.52)
!pip install geopy
!pip install geocoder
Requirement already satisfied: geopy in d:\programdata\anaconda3\lib\site-packages (2.2.0)
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA'
Requirement already satisfied: geographiclib<2,>=1.49 in d:\programdata\anaconda3\lib\site-packages (from geopy) (1.52)
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Collecting geocoder Downloading geocoder-1.38.1-py2.py3-none-any.whl (98 kB) Requirement already satisfied: six in d:\programdata\anaconda3\lib\site-packages (from geocoder) (1.16.0) Requirement already satisfied: requests in d:\programdata\anaconda3\lib\site-packages (from geocoder) (2.27.1) Collecting ratelim Downloading ratelim-0.1.6-py2.py3-none-any.whl (4.0 kB) Requirement already satisfied: future in d:\programdata\anaconda3\lib\site-packages (from geocoder) (0.18.2) Requirement already satisfied: click in d:\programdata\anaconda3\lib\site-packages (from geocoder) (8.0.3) Requirement already satisfied: colorama in d:\programdata\anaconda3\lib\site-packages (from click->geocoder) (0.4.4) Requirement already satisfied: decorator in d:\programdata\anaconda3\lib\site-packages (from ratelim->geocoder) (5.1.1) Requirement already satisfied: idna<4,>=2.5 in d:\programdata\anaconda3\lib\site-packages (from requests->geocoder) (3.3) Requirement already satisfied: certifi>=2017.4.17 in d:\programdata\anaconda3\lib\site-packages (from requests->geocoder) (2021.10.8) Requirement already satisfied: charset-normalizer~=2.0.0 in d:\programdata\anaconda3\lib\site-packages (from requests->geocoder) (2.0.4) Requirement already satisfied: urllib3<1.27,>=1.21.1 in d:\programdata\anaconda3\lib\site-packages (from requests->geocoder) (1.26.8) Installing collected packages: ratelim, geocoder Successfully installed geocoder-1.38.1 ratelim-0.1.6
from geopy.geocoders import Nominatim
geolocator = Nominatim(user_agent="my-my-application")
location = geolocator.geocode("Балтийск Россия")
print(location.address)
#Flatiron Building, 175, 5th Avenue, Flatiron, New York, NYC, New York, ...
print((location.latitude, location.longitude))
#(40.7410861, -73.9896297241625)
Балтийск, Балтийский городской округ, Калининградская область, Северо-Западный федеральный округ, Россия (54.6437214, 19.8941584)
from geopy.geocoders import Nominatim
geolocator = Nominatim(user_agent="my-my-application")
location = geolocator.geocode("Калининград Россия")
print(location.address)
#Flatiron Building, 175, 5th Avenue, Flatiron, New York, NYC, New York, ...
print((location.latitude, location.longitude))
#(40.7410861, -73.9896297241625)
Калининград, городской округ Калининград, Калининградская область, Северо-Западный федеральный округ, Россия (54.710128, 20.5105838)
def city_state_country(row):
coord = f"{row['Latitude']}, {row['Longitude']}"
location = geolocator.reverse(coord, exactly_one=True)
address = location.raw['address']
city = address.get('city', '')
state = address.get('state', '')
country = address.get('country', '')
row['city'] = city
row['state'] = state
row['country'] = country
return row
row = {'Latitude': 54.64, 'Longitude': 19.89}
row = {'Latitude': 55.644466, 'Longitude': 37.395744}
city_state_country(row)
{'Latitude': 55.644466,
'Longitude': 37.395744,
'city': 'Москва',
'state': 'Москва',
'country': 'Россия'}
import geocoder
g = geocoder.osm([51.5074, 0.1278], method='reverse')
g.json['city']
'London'
import geocoder
g = geocoder.osm([54.6437214, 19.8941584], method='reverse')
#str(g).split(',').strip()
print(g.json['town'])
g.json
Балтийск
{'accuracy': 0.001,
'address': 'Администрация Балтийского муниципального района, 6, проспект Ленина, Балтийск, Балтийский городской округ, Калининградская область, Северо-Западный федеральный округ, 238520, Россия',
'bbox': {'northeast': [54.6435977, 19.894209],
'southwest': [54.6429427, 19.8938801]},
'confidence': 10,
'country': 'Россия',
'country_code': 'ru',
'county': 'Балтийский городской округ',
'housenumber': '6',
'importance': 0.001,
'lat': 54.6431757,
'lng': 19.8940599350598,
'ok': True,
'osm_id': 170844680,
'osm_type': 'way',
'place_id': 145593510,
'place_rank': 30,
'postal': '238520',
'quality': 'townhall',
'raw': {'place_id': 145593510,
'licence': 'Data © OpenStreetMap contributors, ODbL 1.0. https://osm.org/copyright',
'osm_type': 'way',
'osm_id': 170844680,
'boundingbox': ['54.6429427', '54.6435977', '19.8938801', '19.894209'],
'lat': '54.6431757',
'lon': '19.8940599350598',
'display_name': 'Администрация Балтийского муниципального района, 6, проспект Ленина, Балтийск, Балтийский городской округ, Калининградская область, Северо-Западный федеральный округ, 238520, Россия',
'place_rank': 30,
'category': 'amenity',
'type': 'townhall',
'importance': 0.001,
'address': {'amenity': 'Администрация Балтийского муниципального района',
'house_number': '6',
'road': 'проспект Ленина',
'town': 'Балтийск',
'county': 'Балтийский городской округ',
'state': 'Калининградская область',
'ISO3166-2-lvl4': 'RU-KGD',
'region': 'Северо-Западный федеральный округ',
'postcode': '238520',
'country': 'Россия',
'country_code': 'ru'}},
'region': 'Калининградская область',
'state': 'Калининградская область',
'status': 'OK',
'street': 'проспект Ленина',
'town': 'Балтийск',
'type': 'townhall'}
!pip install meteostat
Collecting meteostat
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Downloading meteostat-1.6.4-py3-none-any.whl (31 kB) Requirement already satisfied: pytz in d:\programdata\anaconda3\lib\site-packages (from meteostat) (2021.3) Requirement already satisfied: pandas>=1.1 in d:\programdata\anaconda3\lib\site-packages (from meteostat) (1.4.1) Requirement already satisfied: numpy in d:\programdata\anaconda3\lib\site-packages (from meteostat) (1.20.3) Requirement already satisfied: python-dateutil>=2.8.1 in d:\programdata\anaconda3\lib\site-packages (from pandas>=1.1->meteostat) (2.8.2) Requirement already satisfied: six>=1.5 in d:\programdata\anaconda3\lib\site-packages (from python-dateutil>=2.8.1->pandas>=1.1->meteostat) (1.16.0) Installing collected packages: meteostat Successfully installed meteostat-1.6.4
# Import Meteostat library and dependencies
from datetime import datetime
import matplotlib.pyplot as plt
from meteostat import Point, Daily
# Set time period
start = datetime(2018, 1, 1)
end = datetime(2018, 12, 31)
# Create Point for Vancouver, BC
location = Point(49.2497, -123.1193, 70)
# Get daily data for 2018
data = Daily(location, start, end)
data = data.fetch()
# Plot line chart including average, minimum and maximum temperature
data.plot(y=['tavg', 'tmin', 'tmax'])
plt.show()
# Import Meteostat library and dependencies
from datetime import datetime
import matplotlib.pyplot as plt
from meteostat import Point, Daily
# Set time period
start = datetime(2018, 1, 1)
end = datetime(2022, 4, 30)
# Create Point for Kaliningrad
location = Point(54.710128, 20.5105838, 0)
# Get daily data for 2018
data = Daily(location, start, end)
data = data.fetch()
# Plot line chart including average, minimum and maximum temperature
data.plot(y=['tavg', 'tmin', 'tmax'])
plt.show()
import requests
import json
def get_city_opendata(city, country):
tmp = 'https://public.opendatasoft.com/api/records/1.0/search/?dataset=worldcitiespop&q=%s&sort=population&facet=country&refine.country=%s'
cmd = tmp % (city, country)
res = requests.get(cmd)
dct = json.loads(res.content)
return dct
# out = dct['records'][0]['fields']
# return out
get_city_opendata('Berlin', 'de')
#{'city': 'berlin',
# 'country': 'de',
# 'region': '16',
# 'geopoint': [52.516667, 13.4],
# 'longitude': 13.4,
# 'latitude': 52.516667,
# 'accentcity': 'Berlin',
# 'population': 3398362}
get_city_opendata('San Francisco', 'us')
#{'city': 'san francisco',
# 'country': 'us',
# 'region': 'CA',
# 'geopoint': [37.775, -122.4183333],
# 'longitude': -122.4183333,
# 'latitude': 37.775,
# 'accentcity': 'San Francisco',
# 'population': 732072}
{'error': 'Unknown dataset: worldcitiespop'}
!pip install qwikidata
Collecting qwikidata
WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Error parsing requirements for jupyter-packaging: [Errno 2] No such file or directory: 'd:\\programdata\\anaconda3\\lib\\site-packages\\jupyter_packaging-0.7.12.dist-info\\METADATA' WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -yflakes (d:\programdata\anaconda3\lib\site-packages) WARNING: Ignoring invalid distribution -ip (d:\programdata\anaconda3\lib\site-packages)
Downloading qwikidata-0.4.1-py3-none-any.whl (24 kB) Requirement already satisfied: mypy-extensions in d:\programdata\anaconda3\lib\site-packages (from qwikidata) (0.4.3) Requirement already satisfied: requests in d:\programdata\anaconda3\lib\site-packages (from qwikidata) (2.27.1) Requirement already satisfied: urllib3<1.27,>=1.21.1 in d:\programdata\anaconda3\lib\site-packages (from requests->qwikidata) (1.26.8) Requirement already satisfied: charset-normalizer~=2.0.0 in d:\programdata\anaconda3\lib\site-packages (from requests->qwikidata) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in d:\programdata\anaconda3\lib\site-packages (from requests->qwikidata) (3.3) Requirement already satisfied: certifi>=2017.4.17 in d:\programdata\anaconda3\lib\site-packages (from requests->qwikidata) (2021.10.8) Installing collected packages: qwikidata Successfully installed qwikidata-0.4.1
import qwikidata
import qwikidata.sparql
def get_city_wikidata(city, country):
query = """
SELECT ?city ?cityLabel ?country ?countryLabel ?population
WHERE
{
?city rdfs:label '%s'@en.
?city wdt:P1082 ?population.
?city wdt:P17 ?country.
?city rdfs:label ?cityLabel.
?country rdfs:label ?countryLabel.
FILTER(LANG(?cityLabel) = "en").
FILTER(LANG(?countryLabel) = "en").
FILTER(CONTAINS(?countryLabel, "%s")).
}
""" % (city, country)
res = qwikidata.sparql.return_sparql_query_results(query)
out = res['results']['bindings'][0]
return out
get_city_wikidata('Berlin', 'Germany')
#{'city': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q64'},
# 'population': {'datatype': 'http://www.w3.org/2001/XMLSchema#decimal',
# 'type': 'literal',
# 'value': '3613495'},
# 'country': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q183'},
# 'cityLabel': {'xml:lang': 'en', 'type': 'literal', 'value': 'Berlin'},
# 'countryLabel': {'xml:lang': 'en', 'type': 'literal', 'value': 'Germany'}}
get_city_wikidata('San Francisco', 'America')
#{'city': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q62'},
# 'population': {'datatype': 'http://www.w3.org/2001/XMLSchema#decimal',
# 'type': 'literal',
# 'value': '805235'},
# 'country': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q30'},
# 'cityLabel': {'xml:lang': 'en', 'type': 'literal', 'value': 'San Francisco'},
# 'countryLabel': {'xml:lang': 'en',
# 'type': 'literal',
# 'value': 'United States of America'}}
get_city_wikidata('Baltiysk', 'Russia')
{'city': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q5660'},
'population': {'datatype': 'http://www.w3.org/2001/XMLSchema#decimal',
'type': 'literal',
'value': '33317'},
'country': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q159'},
'cityLabel': {'xml:lang': 'en', 'type': 'literal', 'value': 'Baltiysk'},
'countryLabel': {'xml:lang': 'en', 'type': 'literal', 'value': 'Russia'}}
get_city_wikidata('Kaliningrad', 'Russia')
{'city': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q1829'},
'population': {'datatype': 'http://www.w3.org/2001/XMLSchema#decimal',
'type': 'literal',
'value': '475056'},
'country': {'type': 'uri', 'value': 'http://www.wikidata.org/entity/Q159'},
'cityLabel': {'xml:lang': 'en', 'type': 'literal', 'value': 'Kaliningrad'},
'countryLabel': {'xml:lang': 'en', 'type': 'literal', 'value': 'Russia'}}
import matplotlib.pyplot as plt
plt.plot(df_uniq_months.sort_values(ascending=True), marker='.')
plt.show()
#Разделение на train/test для локального тестирования
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=1)
#Создание объекта данных Pool, плюсы: возможность указать какие признаки являются категориальными
pool_train = Pool(X_train, y_train, cat_features = ['PATIENT_SEX', 'MKB_CODE', 'ADRES', 'VISIT_MONTH_YEAR', 'AGE_CATEGORY'])
pool_test = Pool(X_test, cat_features = ['PATIENT_SEX', 'MKB_CODE', 'ADRES', 'VISIT_MONTH_YEAR', 'AGE_CATEGORY'])
#Объявление CatBoostRegressor и обучение
#model = CatBoostRegressor(task_type='GPU')
model = CatBoostRegressor(task_type='CPU')
model.fit(pool_train)
Learning rate set to 0.133461 0: learn: 55.2804763 total: 1.36s remaining: 22m 39s 1: learn: 52.1958994 total: 1.76s remaining: 14m 36s 2: learn: 49.7512641 total: 2.14s remaining: 11m 51s 3: learn: 47.8427742 total: 2.59s remaining: 10m 45s 4: learn: 45.1066612 total: 3.57s remaining: 11m 51s 5: learn: 42.8729630 total: 4.18s remaining: 11m 33s 6: learn: 41.0350387 total: 4.57s remaining: 10m 48s 7: learn: 40.0790438 total: 5.36s remaining: 11m 4s 8: learn: 38.8280294 total: 6.02s remaining: 11m 2s 9: learn: 37.9354252 total: 6.52s remaining: 10m 45s 10: learn: 37.0728095 total: 6.91s remaining: 10m 20s 11: learn: 36.3250217 total: 7.51s remaining: 10m 18s 12: learn: 35.8154001 total: 7.89s remaining: 9m 59s 13: learn: 35.4129958 total: 8.33s remaining: 9m 47s 14: learn: 35.0779532 total: 8.72s remaining: 9m 32s 15: learn: 34.7130334 total: 9.19s remaining: 9m 25s 16: learn: 34.3891154 total: 9.58s remaining: 9m 13s 17: learn: 34.0864163 total: 9.97s remaining: 9m 4s 18: learn: 33.8058898 total: 10.9s remaining: 9m 24s 19: learn: 33.5946650 total: 11.8s remaining: 9m 36s 20: learn: 33.4249243 total: 12.1s remaining: 9m 25s 21: learn: 33.2240767 total: 12.6s remaining: 9m 19s 22: learn: 33.0212507 total: 13s remaining: 9m 13s 23: learn: 32.8305867 total: 13.9s remaining: 9m 26s 24: learn: 32.3760585 total: 14.4s remaining: 9m 22s 25: learn: 32.1905885 total: 14.8s remaining: 9m 14s 26: learn: 31.8895444 total: 15.4s remaining: 9m 16s 27: learn: 31.5649969 total: 15.9s remaining: 9m 10s 28: learn: 31.4757653 total: 16.3s remaining: 9m 4s 29: learn: 31.1685824 total: 16.7s remaining: 8m 59s 30: learn: 30.9124750 total: 17.6s remaining: 9m 9s 31: learn: 30.7686622 total: 18s remaining: 9m 5s 32: learn: 30.4766965 total: 18.4s remaining: 8m 59s 33: learn: 30.3234860 total: 19.1s remaining: 9m 3s 34: learn: 30.2398800 total: 19.7s remaining: 9m 2s 35: learn: 30.0004732 total: 20.5s remaining: 9m 7s 36: learn: 29.9632992 total: 20.9s remaining: 9m 2s 37: learn: 29.7179571 total: 21.2s remaining: 8m 57s 38: learn: 29.5621486 total: 21.6s remaining: 8m 53s 39: learn: 29.3560742 total: 22s remaining: 8m 49s 40: learn: 29.2172834 total: 22.4s remaining: 8m 44s 41: learn: 29.1858736 total: 22.9s remaining: 8m 41s 42: learn: 29.1648001 total: 23.3s remaining: 8m 37s 43: learn: 29.1259811 total: 23.7s remaining: 8m 33s 44: learn: 28.9406437 total: 24s remaining: 8m 29s 45: learn: 28.9154104 total: 24.5s remaining: 8m 27s 46: learn: 28.7742416 total: 24.8s remaining: 8m 23s 47: learn: 28.6631192 total: 25.2s remaining: 8m 19s 48: learn: 28.5454089 total: 25.6s remaining: 8m 16s 49: learn: 28.4918051 total: 25.9s remaining: 8m 12s 50: learn: 28.3137395 total: 26.3s remaining: 8m 9s 51: learn: 28.2175734 total: 26.7s remaining: 8m 7s 52: learn: 28.1461910 total: 27.1s remaining: 8m 4s 53: learn: 28.0706708 total: 27.5s remaining: 8m 2s 54: learn: 27.9850814 total: 27.9s remaining: 7m 59s 55: learn: 27.9599371 total: 28.3s remaining: 7m 56s 56: learn: 27.8951805 total: 28.6s remaining: 7m 53s 57: learn: 27.7584948 total: 29s remaining: 7m 51s 58: learn: 27.6762792 total: 29.6s remaining: 7m 52s 59: learn: 27.6680684 total: 30s remaining: 7m 50s 60: learn: 27.6680682 total: 30.2s remaining: 7m 45s 61: learn: 27.5230087 total: 30.6s remaining: 7m 43s 62: learn: 27.4396929 total: 31s remaining: 7m 41s 63: learn: 27.4396671 total: 31.3s remaining: 7m 37s 64: learn: 27.3319171 total: 31.7s remaining: 7m 35s 65: learn: 27.2905857 total: 32.1s remaining: 7m 33s 66: learn: 27.2652544 total: 32.4s remaining: 7m 31s 67: learn: 27.2125084 total: 32.8s remaining: 7m 29s 68: learn: 27.1833727 total: 33.2s remaining: 7m 28s 69: learn: 27.0699021 total: 33.6s remaining: 7m 26s 70: learn: 27.0386957 total: 34s remaining: 7m 24s 71: learn: 26.9755474 total: 34.4s remaining: 7m 22s 72: learn: 26.8820630 total: 34.8s remaining: 7m 21s 73: learn: 26.8191153 total: 35.3s remaining: 7m 21s 74: learn: 26.7592873 total: 35.6s remaining: 7m 19s 75: learn: 26.7511438 total: 36s remaining: 7m 17s 76: learn: 26.7044227 total: 36.4s remaining: 7m 16s 77: learn: 26.7012594 total: 36.8s remaining: 7m 14s 78: learn: 26.5397876 total: 37.2s remaining: 7m 13s 79: learn: 26.5242451 total: 37.5s remaining: 7m 11s 80: learn: 26.4684037 total: 37.9s remaining: 7m 10s 81: learn: 26.3414680 total: 38.5s remaining: 7m 11s 82: learn: 26.2998543 total: 38.9s remaining: 7m 9s 83: learn: 26.2103493 total: 39.3s remaining: 7m 8s 84: learn: 26.1838323 total: 39.7s remaining: 7m 7s 85: learn: 26.1633051 total: 40.1s remaining: 7m 5s 86: learn: 26.0735983 total: 40.4s remaining: 7m 4s 87: learn: 25.9887715 total: 40.8s remaining: 7m 3s 88: learn: 25.9521706 total: 41.2s remaining: 7m 2s 89: learn: 25.9175313 total: 41.6s remaining: 7m 90: learn: 25.8208352 total: 42s remaining: 6m 59s 91: learn: 25.7963761 total: 42.4s remaining: 6m 58s 92: learn: 25.7302575 total: 42.8s remaining: 6m 57s 93: learn: 25.6758552 total: 43.3s remaining: 6m 57s 94: learn: 25.5542159 total: 43.7s remaining: 6m 56s 95: learn: 25.5029620 total: 44.1s remaining: 6m 54s 96: learn: 25.4891731 total: 44.5s remaining: 6m 53s 97: learn: 25.4443812 total: 44.8s remaining: 6m 52s 98: learn: 25.4395160 total: 45.2s remaining: 6m 51s 99: learn: 25.4384308 total: 45.6s remaining: 6m 50s 100: learn: 25.3921562 total: 45.9s remaining: 6m 48s 101: learn: 25.3775915 total: 46.3s remaining: 6m 47s 102: learn: 25.3556863 total: 46.7s remaining: 6m 46s 103: learn: 25.2992178 total: 47.1s remaining: 6m 45s 104: learn: 25.2616521 total: 47.4s remaining: 6m 44s 105: learn: 25.2400130 total: 47.8s remaining: 6m 43s 106: learn: 25.1887264 total: 48.2s remaining: 6m 42s 107: learn: 25.1828889 total: 48.6s remaining: 6m 41s 108: learn: 25.1104981 total: 49s remaining: 6m 40s 109: learn: 25.0443663 total: 49.3s remaining: 6m 39s 110: learn: 24.9877812 total: 49.7s remaining: 6m 38s 111: learn: 24.9426347 total: 50.1s remaining: 6m 37s 112: learn: 24.9306727 total: 50.5s remaining: 6m 36s 113: learn: 24.9173952 total: 50.9s remaining: 6m 35s 114: learn: 24.8887975 total: 51.3s remaining: 6m 34s 115: learn: 24.8495157 total: 51.6s remaining: 6m 33s 116: learn: 24.8274884 total: 52s remaining: 6m 32s 117: learn: 24.7707769 total: 52.4s remaining: 6m 31s 118: learn: 24.7284341 total: 52.8s remaining: 6m 30s 119: learn: 24.7196407 total: 53.1s remaining: 6m 29s 120: learn: 24.7040187 total: 53.5s remaining: 6m 28s 121: learn: 24.6917767 total: 53.9s remaining: 6m 27s 122: learn: 24.6857682 total: 54.3s remaining: 6m 27s 123: learn: 24.6744737 total: 54.7s remaining: 6m 26s 124: learn: 24.6606174 total: 55.2s remaining: 6m 26s 125: learn: 24.6305267 total: 55.6s remaining: 6m 25s 126: learn: 24.6007203 total: 56s remaining: 6m 24s 127: learn: 24.5510997 total: 56.4s remaining: 6m 24s 128: learn: 24.5265587 total: 56.8s remaining: 6m 23s 129: learn: 24.4358379 total: 57.3s remaining: 6m 23s 130: learn: 24.4169810 total: 57.8s remaining: 6m 23s 131: learn: 24.3780805 total: 58.3s remaining: 6m 23s 132: learn: 24.2987804 total: 58.6s remaining: 6m 22s 133: learn: 24.2939746 total: 59s remaining: 6m 21s 134: learn: 24.2534233 total: 59.4s remaining: 6m 20s 135: learn: 24.2181954 total: 59.8s remaining: 6m 20s 136: learn: 24.1907702 total: 1m remaining: 6m 19s 137: learn: 24.1589328 total: 1m remaining: 6m 18s 138: learn: 24.1479418 total: 1m remaining: 6m 17s 139: learn: 24.1299003 total: 1m 1s remaining: 6m 17s 140: learn: 24.0671771 total: 1m 1s remaining: 6m 16s 141: learn: 24.0361314 total: 1m 2s remaining: 6m 16s 142: learn: 23.9982823 total: 1m 2s remaining: 6m 15s 143: learn: 23.9893427 total: 1m 3s remaining: 6m 15s 144: learn: 23.9795322 total: 1m 3s remaining: 6m 14s 145: learn: 23.8977299 total: 1m 3s remaining: 6m 13s 146: learn: 23.8662208 total: 1m 4s remaining: 6m 12s 147: learn: 23.8345845 total: 1m 4s remaining: 6m 12s 148: learn: 23.7955083 total: 1m 5s remaining: 6m 11s 149: learn: 23.7519556 total: 1m 5s remaining: 6m 11s 150: learn: 23.7189313 total: 1m 5s remaining: 6m 10s 151: learn: 23.6822690 total: 1m 6s remaining: 6m 9s 152: learn: 23.6774545 total: 1m 6s remaining: 6m 8s 153: learn: 23.6716021 total: 1m 6s remaining: 6m 7s 154: learn: 23.6430297 total: 1m 7s remaining: 6m 7s 155: learn: 23.6393238 total: 1m 7s remaining: 6m 6s 156: learn: 23.5746138 total: 1m 8s remaining: 6m 5s 157: learn: 23.5531796 total: 1m 8s remaining: 6m 5s 158: learn: 23.5524074 total: 1m 8s remaining: 6m 4s 159: learn: 23.5126942 total: 1m 9s remaining: 6m 3s 160: learn: 23.4930295 total: 1m 9s remaining: 6m 2s 161: learn: 23.4675704 total: 1m 9s remaining: 6m 1s 162: learn: 23.4351118 total: 1m 10s remaining: 6m 1s 163: learn: 23.4294030 total: 1m 10s remaining: 6m 164: learn: 23.3984916 total: 1m 11s remaining: 5m 59s 165: learn: 23.3682460 total: 1m 11s remaining: 5m 59s 166: learn: 23.3369970 total: 1m 11s remaining: 5m 58s 167: learn: 23.3023950 total: 1m 12s remaining: 5m 57s 168: learn: 23.2677426 total: 1m 12s remaining: 5m 57s 169: learn: 23.2403413 total: 1m 12s remaining: 5m 56s 170: learn: 23.2233708 total: 1m 13s remaining: 5m 55s 171: learn: 23.2084993 total: 1m 13s remaining: 5m 54s 172: learn: 23.1721168 total: 1m 14s remaining: 5m 54s 173: learn: 23.1088380 total: 1m 14s remaining: 5m 53s 174: learn: 23.0890510 total: 1m 14s remaining: 5m 52s 175: learn: 23.0843081 total: 1m 15s remaining: 5m 51s 176: learn: 23.0171561 total: 1m 15s remaining: 5m 51s 177: learn: 22.9977189 total: 1m 15s remaining: 5m 50s 178: learn: 22.9917384 total: 1m 16s remaining: 5m 50s 179: learn: 22.9792782 total: 1m 16s remaining: 5m 49s 180: learn: 22.9650673 total: 1m 17s remaining: 5m 48s 181: learn: 22.9460036 total: 1m 17s remaining: 5m 48s 182: learn: 22.9355602 total: 1m 17s remaining: 5m 47s 183: learn: 22.9259264 total: 1m 18s remaining: 5m 47s 184: learn: 22.9078775 total: 1m 18s remaining: 5m 47s 185: learn: 22.8990385 total: 1m 19s remaining: 5m 46s 186: learn: 22.8775337 total: 1m 19s remaining: 5m 46s 187: learn: 22.8454074 total: 1m 20s remaining: 5m 46s 188: learn: 22.8265520 total: 1m 20s remaining: 5m 45s 189: learn: 22.7770235 total: 1m 21s remaining: 5m 45s 190: learn: 22.7696133 total: 1m 21s remaining: 5m 44s 191: learn: 22.7657588 total: 1m 21s remaining: 5m 44s 192: learn: 22.7531983 total: 1m 22s remaining: 5m 43s 193: learn: 22.7381519 total: 1m 22s remaining: 5m 43s 194: learn: 22.7163971 total: 1m 23s remaining: 5m 42s 195: learn: 22.6955602 total: 1m 23s remaining: 5m 42s 196: learn: 22.6930276 total: 1m 23s remaining: 5m 41s 197: learn: 22.6769800 total: 1m 24s remaining: 5m 41s 198: learn: 22.6687181 total: 1m 24s remaining: 5m 41s 199: learn: 22.6350642 total: 1m 25s remaining: 5m 40s 200: learn: 22.5949248 total: 1m 25s remaining: 5m 40s 201: learn: 22.5304894 total: 1m 25s remaining: 5m 39s 202: learn: 22.4737816 total: 1m 26s remaining: 5m 39s 203: learn: 22.4586675 total: 1m 26s remaining: 5m 38s 204: learn: 22.4392550 total: 1m 27s remaining: 5m 37s 205: learn: 22.4029166 total: 1m 27s remaining: 5m 37s 206: learn: 22.3896882 total: 1m 27s remaining: 5m 36s 207: learn: 22.3683488 total: 1m 28s remaining: 5m 36s 208: learn: 22.3607242 total: 1m 28s remaining: 5m 35s 209: learn: 22.3548694 total: 1m 29s remaining: 5m 35s 210: learn: 22.3264088 total: 1m 29s remaining: 5m 34s 211: learn: 22.3059973 total: 1m 29s remaining: 5m 33s 212: learn: 22.2882074 total: 1m 30s remaining: 5m 33s 213: learn: 22.2715885 total: 1m 30s remaining: 5m 32s 214: learn: 22.2513976 total: 1m 30s remaining: 5m 32s 215: learn: 22.2370211 total: 1m 31s remaining: 5m 31s 216: learn: 22.2312107 total: 1m 31s remaining: 5m 31s 217: learn: 22.2267941 total: 1m 32s remaining: 5m 30s 218: learn: 22.2212492 total: 1m 32s remaining: 5m 29s 219: learn: 22.2160656 total: 1m 32s remaining: 5m 29s 220: learn: 22.2128794 total: 1m 33s remaining: 5m 28s 221: learn: 22.1955781 total: 1m 33s remaining: 5m 27s 222: learn: 22.1886017 total: 1m 33s remaining: 5m 27s 223: learn: 22.1833587 total: 1m 34s remaining: 5m 26s 224: learn: 22.1549391 total: 1m 34s remaining: 5m 26s 225: learn: 22.1274632 total: 1m 35s remaining: 5m 25s 226: learn: 22.1156565 total: 1m 35s remaining: 5m 25s 227: learn: 22.0652480 total: 1m 35s remaining: 5m 24s 228: learn: 22.0610453 total: 1m 36s remaining: 5m 24s 229: learn: 22.0294263 total: 1m 36s remaining: 5m 23s 230: learn: 22.0066253 total: 1m 37s remaining: 5m 23s 231: learn: 21.9855126 total: 1m 37s remaining: 5m 22s 232: learn: 21.9611421 total: 1m 37s remaining: 5m 22s 233: learn: 21.9479826 total: 1m 38s remaining: 5m 21s 234: learn: 21.9378731 total: 1m 38s remaining: 5m 21s 235: learn: 21.9273260 total: 1m 39s remaining: 5m 20s 236: learn: 21.9135138 total: 1m 39s remaining: 5m 20s 237: learn: 21.9080279 total: 1m 39s remaining: 5m 19s 238: learn: 21.9046330 total: 1m 40s remaining: 5m 19s 239: learn: 21.9017928 total: 1m 40s remaining: 5m 18s 240: learn: 21.8794484 total: 1m 41s remaining: 5m 18s 241: learn: 21.8501840 total: 1m 41s remaining: 5m 17s 242: learn: 21.8473649 total: 1m 41s remaining: 5m 17s 243: learn: 21.8393525 total: 1m 42s remaining: 5m 16s 244: learn: 21.8221055 total: 1m 42s remaining: 5m 16s 245: learn: 21.8146934 total: 1m 42s remaining: 5m 15s 246: learn: 21.8103101 total: 1m 43s remaining: 5m 14s 247: learn: 21.8001571 total: 1m 43s remaining: 5m 14s 248: learn: 21.7993393 total: 1m 44s remaining: 5m 13s 249: learn: 21.7871839 total: 1m 44s remaining: 5m 13s 250: learn: 21.7839855 total: 1m 44s remaining: 5m 12s 251: learn: 21.7554741 total: 1m 45s remaining: 5m 12s 252: learn: 21.7446125 total: 1m 45s remaining: 5m 12s 253: learn: 21.7334101 total: 1m 46s remaining: 5m 11s 254: learn: 21.7246225 total: 1m 46s remaining: 5m 11s 255: learn: 21.7137759 total: 1m 46s remaining: 5m 10s 256: learn: 21.7070606 total: 1m 47s remaining: 5m 10s 257: learn: 21.6920039 total: 1m 47s remaining: 5m 9s 258: learn: 21.6904105 total: 1m 48s remaining: 5m 9s 259: learn: 21.6880983 total: 1m 48s remaining: 5m 8s 260: learn: 21.6815198 total: 1m 48s remaining: 5m 8s 261: learn: 21.6769181 total: 1m 49s remaining: 5m 7s 262: learn: 21.6435755 total: 1m 49s remaining: 5m 7s 263: learn: 21.6027135 total: 1m 50s remaining: 5m 6s 264: learn: 21.5993703 total: 1m 50s remaining: 5m 6s 265: learn: 21.5786371 total: 1m 50s remaining: 5m 5s 266: learn: 21.5273172 total: 1m 51s remaining: 5m 5s 267: learn: 21.5240061 total: 1m 51s remaining: 5m 4s 268: learn: 21.5160570 total: 1m 51s remaining: 5m 4s 269: learn: 21.5014634 total: 1m 52s remaining: 5m 3s 270: learn: 21.4903771 total: 1m 52s remaining: 5m 3s 271: learn: 21.4826619 total: 1m 53s remaining: 5m 2s 272: learn: 21.4760945 total: 1m 53s remaining: 5m 2s 273: learn: 21.4716068 total: 1m 53s remaining: 5m 1s 274: learn: 21.4624785 total: 1m 54s remaining: 5m 1s 275: learn: 21.4593496 total: 1m 54s remaining: 5m 276: learn: 21.4537135 total: 1m 55s remaining: 5m 277: learn: 21.4386782 total: 1m 55s remaining: 5m 278: learn: 21.4204107 total: 1m 56s remaining: 4m 59s 279: learn: 21.4130642 total: 1m 56s remaining: 4m 59s 280: learn: 21.3981870 total: 1m 56s remaining: 4m 59s 281: learn: 21.3950238 total: 1m 57s remaining: 4m 58s 282: learn: 21.3896224 total: 1m 57s remaining: 4m 58s 283: learn: 21.3757838 total: 1m 58s remaining: 4m 57s 284: learn: 21.3656041 total: 1m 58s remaining: 4m 57s 285: learn: 21.3553070 total: 1m 58s remaining: 4m 56s 286: learn: 21.3506662 total: 1m 59s remaining: 4m 56s 287: learn: 21.3368825 total: 1m 59s remaining: 4m 55s 288: learn: 21.3299791 total: 1m 59s remaining: 4m 55s 289: learn: 21.3203368 total: 2m remaining: 4m 54s 290: learn: 21.3149770 total: 2m remaining: 4m 54s 291: learn: 21.3043199 total: 2m 1s remaining: 4m 53s 292: learn: 21.3020164 total: 2m 1s remaining: 4m 53s 293: learn: 21.2950135 total: 2m 1s remaining: 4m 52s 294: learn: 21.2896131 total: 2m 2s remaining: 4m 52s 295: learn: 21.2881545 total: 2m 2s remaining: 4m 51s 296: learn: 21.2858335 total: 2m 3s remaining: 4m 51s 297: learn: 21.2778914 total: 2m 3s remaining: 4m 50s 298: learn: 21.2768068 total: 2m 3s remaining: 4m 50s 299: learn: 21.2616206 total: 2m 4s remaining: 4m 49s 300: learn: 21.2577706 total: 2m 4s remaining: 4m 49s 301: learn: 21.2411495 total: 2m 4s remaining: 4m 48s 302: learn: 21.2371865 total: 2m 5s remaining: 4m 48s 303: learn: 21.2068967 total: 2m 5s remaining: 4m 48s 304: learn: 21.1964181 total: 2m 6s remaining: 4m 47s 305: learn: 21.1960447 total: 2m 6s remaining: 4m 47s 306: learn: 21.1847288 total: 2m 6s remaining: 4m 46s 307: learn: 21.1844616 total: 2m 7s remaining: 4m 46s 308: learn: 21.1800263 total: 2m 7s remaining: 4m 45s 309: learn: 21.1527098 total: 2m 8s remaining: 4m 45s 310: learn: 21.1426999 total: 2m 8s remaining: 4m 44s 311: learn: 21.1383675 total: 2m 9s remaining: 4m 44s 312: learn: 21.1326843 total: 2m 9s remaining: 4m 44s 313: learn: 21.1155816 total: 2m 9s remaining: 4m 43s 314: learn: 21.1110173 total: 2m 10s remaining: 4m 43s 315: learn: 21.1077223 total: 2m 10s remaining: 4m 42s 316: learn: 21.0635373 total: 2m 11s remaining: 4m 42s 317: learn: 21.0517261 total: 2m 11s remaining: 4m 42s 318: learn: 21.0485337 total: 2m 12s remaining: 4m 41s 319: learn: 21.0314858 total: 2m 12s remaining: 4m 41s 320: learn: 21.0264568 total: 2m 12s remaining: 4m 40s 321: learn: 21.0198580 total: 2m 13s remaining: 4m 40s 322: learn: 21.0125547 total: 2m 13s remaining: 4m 40s 323: learn: 21.0033877 total: 2m 14s remaining: 4m 39s 324: learn: 20.9896495 total: 2m 14s remaining: 4m 39s 325: learn: 20.9630581 total: 2m 14s remaining: 4m 38s 326: learn: 20.9511615 total: 2m 15s remaining: 4m 38s 327: learn: 20.9485979 total: 2m 15s remaining: 4m 37s 328: learn: 20.9469548 total: 2m 15s remaining: 4m 37s 329: learn: 20.9381924 total: 2m 16s remaining: 4m 36s 330: learn: 20.9357977 total: 2m 16s remaining: 4m 36s 331: learn: 20.9311682 total: 2m 17s remaining: 4m 35s 332: learn: 20.9261250 total: 2m 17s remaining: 4m 35s 333: learn: 20.9129182 total: 2m 17s remaining: 4m 35s 334: learn: 20.9037508 total: 2m 18s remaining: 4m 34s 335: learn: 20.9016550 total: 2m 18s remaining: 4m 34s 336: learn: 20.8942033 total: 2m 19s remaining: 4m 33s 337: learn: 20.8933823 total: 2m 19s remaining: 4m 33s 338: learn: 20.8932915 total: 2m 19s remaining: 4m 32s 339: learn: 20.8909367 total: 2m 20s remaining: 4m 32s 340: learn: 20.8881710 total: 2m 20s remaining: 4m 31s 341: learn: 20.8648048 total: 2m 21s remaining: 4m 31s 342: learn: 20.8571409 total: 2m 21s remaining: 4m 30s 343: learn: 20.8473724 total: 2m 21s remaining: 4m 30s 344: learn: 20.8338361 total: 2m 22s remaining: 4m 29s 345: learn: 20.8310427 total: 2m 22s remaining: 4m 29s 346: learn: 20.7961893 total: 2m 22s remaining: 4m 28s 347: learn: 20.7925235 total: 2m 23s remaining: 4m 28s 348: learn: 20.7846600 total: 2m 23s remaining: 4m 27s 349: learn: 20.7380625 total: 2m 24s remaining: 4m 27s 350: learn: 20.7377977 total: 2m 24s remaining: 4m 27s 351: learn: 20.7345615 total: 2m 24s remaining: 4m 26s 352: learn: 20.7223704 total: 2m 25s remaining: 4m 26s 353: learn: 20.7193333 total: 2m 25s remaining: 4m 25s 354: learn: 20.7095797 total: 2m 25s remaining: 4m 25s 355: learn: 20.7013686 total: 2m 26s remaining: 4m 24s 356: learn: 20.6996084 total: 2m 26s remaining: 4m 24s 357: learn: 20.6946358 total: 2m 27s remaining: 4m 23s 358: learn: 20.6929575 total: 2m 27s remaining: 4m 23s 359: learn: 20.6887881 total: 2m 27s remaining: 4m 22s 360: learn: 20.6876282 total: 2m 28s remaining: 4m 22s 361: learn: 20.6618329 total: 2m 28s remaining: 4m 21s 362: learn: 20.6557705 total: 2m 28s remaining: 4m 21s 363: learn: 20.6449169 total: 2m 29s remaining: 4m 20s 364: learn: 20.6293495 total: 2m 29s remaining: 4m 20s 365: learn: 20.6276147 total: 2m 30s remaining: 4m 20s 366: learn: 20.6268166 total: 2m 30s remaining: 4m 19s 367: learn: 20.6264281 total: 2m 30s remaining: 4m 19s 368: learn: 20.6070569 total: 2m 31s remaining: 4m 18s 369: learn: 20.5898769 total: 2m 31s remaining: 4m 18s 370: learn: 20.5570370 total: 2m 32s remaining: 4m 17s 371: learn: 20.5505399 total: 2m 32s remaining: 4m 17s 372: learn: 20.5400048 total: 2m 32s remaining: 4m 16s 373: learn: 20.5326805 total: 2m 33s remaining: 4m 16s 374: learn: 20.5302511 total: 2m 33s remaining: 4m 16s 375: learn: 20.5300419 total: 2m 34s remaining: 4m 15s 376: learn: 20.5223078 total: 2m 34s remaining: 4m 15s 377: learn: 20.5174921 total: 2m 34s remaining: 4m 14s 378: learn: 20.5146586 total: 2m 35s remaining: 4m 14s 379: learn: 20.4983364 total: 2m 35s remaining: 4m 13s 380: learn: 20.4947296 total: 2m 36s remaining: 4m 13s 381: learn: 20.4869834 total: 2m 36s remaining: 4m 13s 382: learn: 20.4823165 total: 2m 36s remaining: 4m 12s 383: learn: 20.4793059 total: 2m 37s remaining: 4m 12s 384: learn: 20.4695664 total: 2m 37s remaining: 4m 11s 385: learn: 20.4691841 total: 2m 37s remaining: 4m 11s 386: learn: 20.4600147 total: 2m 38s remaining: 4m 10s 387: learn: 20.4598227 total: 2m 38s remaining: 4m 10s 388: learn: 20.4444272 total: 2m 39s remaining: 4m 10s 389: learn: 20.4397344 total: 2m 39s remaining: 4m 9s 390: learn: 20.4394327 total: 2m 39s remaining: 4m 9s 391: learn: 20.4256976 total: 2m 40s remaining: 4m 8s 392: learn: 20.4234029 total: 2m 40s remaining: 4m 8s 393: learn: 20.4195907 total: 2m 41s remaining: 4m 7s 394: learn: 20.4175476 total: 2m 41s remaining: 4m 7s 395: learn: 20.4110166 total: 2m 41s remaining: 4m 6s 396: learn: 20.4077276 total: 2m 42s remaining: 4m 6s 397: learn: 20.4074138 total: 2m 42s remaining: 4m 5s 398: learn: 20.4018383 total: 2m 42s remaining: 4m 5s 399: learn: 20.3782053 total: 2m 43s remaining: 4m 5s 400: learn: 20.3650883 total: 2m 43s remaining: 4m 4s 401: learn: 20.3534392 total: 2m 44s remaining: 4m 4s 402: learn: 20.3525187 total: 2m 44s remaining: 4m 3s 403: learn: 20.3524681 total: 2m 44s remaining: 4m 3s 404: learn: 20.3373549 total: 2m 45s remaining: 4m 2s 405: learn: 20.3345146 total: 2m 45s remaining: 4m 2s 406: learn: 20.3342520 total: 2m 46s remaining: 4m 1s 407: learn: 20.3097714 total: 2m 46s remaining: 4m 1s 408: learn: 20.3095664 total: 2m 46s remaining: 4m 409: learn: 20.2988839 total: 2m 47s remaining: 4m 410: learn: 20.2951136 total: 2m 47s remaining: 4m 411: learn: 20.2950538 total: 2m 47s remaining: 3m 59s 412: learn: 20.2928730 total: 2m 48s remaining: 3m 59s 413: learn: 20.2893132 total: 2m 48s remaining: 3m 58s 414: learn: 20.2851020 total: 2m 49s remaining: 3m 58s 415: learn: 20.2744312 total: 2m 49s remaining: 3m 57s 416: learn: 20.2650351 total: 2m 49s remaining: 3m 57s 417: learn: 20.2554677 total: 2m 50s remaining: 3m 57s 418: learn: 20.2465822 total: 2m 50s remaining: 3m 56s 419: learn: 20.2401908 total: 2m 50s remaining: 3m 56s 420: learn: 20.2290609 total: 2m 51s remaining: 3m 55s 421: learn: 20.2239537 total: 2m 51s remaining: 3m 55s 422: learn: 20.2229762 total: 2m 52s remaining: 3m 54s 423: learn: 20.2222912 total: 2m 52s remaining: 3m 54s 424: learn: 20.2196016 total: 2m 52s remaining: 3m 54s 425: learn: 20.2131779 total: 2m 53s remaining: 3m 53s 426: learn: 20.2102062 total: 2m 53s remaining: 3m 53s 427: learn: 20.2038234 total: 2m 54s remaining: 3m 52s 428: learn: 20.1940056 total: 2m 54s remaining: 3m 52s 429: learn: 20.1882704 total: 2m 55s remaining: 3m 52s 430: learn: 20.1852486 total: 2m 55s remaining: 3m 51s 431: learn: 20.1651999 total: 2m 55s remaining: 3m 51s 432: learn: 20.1630263 total: 2m 56s remaining: 3m 50s 433: learn: 20.1610587 total: 2m 57s remaining: 3m 51s 434: learn: 20.1564221 total: 2m 57s remaining: 3m 50s 435: learn: 20.1544298 total: 2m 57s remaining: 3m 50s 436: learn: 20.1541528 total: 2m 58s remaining: 3m 49s 437: learn: 20.1539461 total: 2m 58s remaining: 3m 49s 438: learn: 20.1401966 total: 2m 59s remaining: 3m 49s 439: learn: 20.1400181 total: 2m 59s remaining: 3m 48s 440: learn: 20.1384144 total: 3m remaining: 3m 48s 441: learn: 20.1364809 total: 3m remaining: 3m 47s 442: learn: 20.1363058 total: 3m remaining: 3m 47s 443: learn: 20.1284352 total: 3m 1s remaining: 3m 46s 444: learn: 20.1282875 total: 3m 1s remaining: 3m 46s 445: learn: 20.1281716 total: 3m 2s remaining: 3m 46s 446: learn: 20.1058847 total: 3m 2s remaining: 3m 45s 447: learn: 20.0817002 total: 3m 2s remaining: 3m 45s 448: learn: 20.0816488 total: 3m 3s remaining: 3m 44s 449: learn: 20.0814062 total: 3m 3s remaining: 3m 44s 450: learn: 20.0776090 total: 3m 4s remaining: 3m 44s 451: learn: 20.0766180 total: 3m 4s remaining: 3m 43s 452: learn: 20.0685439 total: 3m 4s remaining: 3m 43s 453: learn: 20.0658463 total: 3m 5s remaining: 3m 42s 454: learn: 20.0643304 total: 3m 5s remaining: 3m 42s 455: learn: 20.0610323 total: 3m 6s remaining: 3m 41s 456: learn: 20.0602814 total: 3m 6s remaining: 3m 41s 457: learn: 20.0568341 total: 3m 6s remaining: 3m 41s 458: learn: 20.0538389 total: 3m 7s remaining: 3m 40s 459: learn: 20.0528169 total: 3m 7s remaining: 3m 40s 460: learn: 20.0381449 total: 3m 7s remaining: 3m 39s 461: learn: 20.0324538 total: 3m 8s remaining: 3m 39s 462: learn: 20.0313902 total: 3m 8s remaining: 3m 38s 463: learn: 20.0291021 total: 3m 9s remaining: 3m 38s 464: learn: 20.0253233 total: 3m 9s remaining: 3m 38s 465: learn: 20.0229387 total: 3m 10s remaining: 3m 37s 466: learn: 20.0133605 total: 3m 10s remaining: 3m 37s 467: learn: 20.0060185 total: 3m 10s remaining: 3m 36s 468: learn: 19.9988033 total: 3m 11s remaining: 3m 36s 469: learn: 19.9971749 total: 3m 11s remaining: 3m 36s 470: learn: 19.9923915 total: 3m 12s remaining: 3m 35s 471: learn: 19.9864255 total: 3m 12s remaining: 3m 35s 472: learn: 19.9858600 total: 3m 12s remaining: 3m 34s 473: learn: 19.9836580 total: 3m 13s remaining: 3m 34s 474: learn: 19.9831907 total: 3m 13s remaining: 3m 33s 475: learn: 19.9813442 total: 3m 13s remaining: 3m 33s 476: learn: 19.9809395 total: 3m 14s remaining: 3m 33s 477: learn: 19.9695180 total: 3m 14s remaining: 3m 32s 478: learn: 19.9633177 total: 3m 15s remaining: 3m 32s 479: learn: 19.9629621 total: 3m 15s remaining: 3m 31s 480: learn: 19.9479688 total: 3m 15s remaining: 3m 31s 481: learn: 19.9383720 total: 3m 16s remaining: 3m 30s 482: learn: 19.9353834 total: 3m 16s remaining: 3m 30s 483: learn: 19.9333378 total: 3m 16s remaining: 3m 29s 484: learn: 19.9293715 total: 3m 17s remaining: 3m 29s 485: learn: 19.9162056 total: 3m 17s remaining: 3m 29s 486: learn: 19.9122363 total: 3m 18s remaining: 3m 28s 487: learn: 19.9112281 total: 3m 18s remaining: 3m 28s 488: learn: 19.9085930 total: 3m 18s remaining: 3m 27s 489: learn: 19.9076211 total: 3m 19s remaining: 3m 27s 490: learn: 19.8679702 total: 3m 19s remaining: 3m 26s 491: learn: 19.8666599 total: 3m 20s remaining: 3m 26s 492: learn: 19.8614280 total: 3m 20s remaining: 3m 26s 493: learn: 19.8328035 total: 3m 20s remaining: 3m 25s 494: learn: 19.8310201 total: 3m 21s remaining: 3m 25s 495: learn: 19.8095151 total: 3m 21s remaining: 3m 24s 496: learn: 19.8085564 total: 3m 22s remaining: 3m 24s 497: learn: 19.8029496 total: 3m 22s remaining: 3m 24s 498: learn: 19.7963930 total: 3m 22s remaining: 3m 23s 499: learn: 19.7836138 total: 3m 23s remaining: 3m 23s 500: learn: 19.7658679 total: 3m 23s remaining: 3m 22s 501: learn: 19.7586225 total: 3m 24s remaining: 3m 22s 502: learn: 19.7536205 total: 3m 24s remaining: 3m 22s 503: learn: 19.7532984 total: 3m 24s remaining: 3m 21s 504: learn: 19.7419765 total: 3m 25s remaining: 3m 21s 505: learn: 19.7324351 total: 3m 25s remaining: 3m 20s 506: learn: 19.7165787 total: 3m 26s remaining: 3m 20s 507: learn: 19.7007395 total: 3m 26s remaining: 3m 20s 508: learn: 19.6742805 total: 3m 27s remaining: 3m 19s 509: learn: 19.6538350 total: 3m 27s remaining: 3m 19s 510: learn: 19.6197507 total: 3m 27s remaining: 3m 18s 511: learn: 19.6146504 total: 3m 28s remaining: 3m 18s 512: learn: 19.6114219 total: 3m 28s remaining: 3m 18s 513: learn: 19.6080033 total: 3m 29s remaining: 3m 17s 514: learn: 19.6026271 total: 3m 29s remaining: 3m 17s 515: learn: 19.6019486 total: 3m 29s remaining: 3m 16s 516: learn: 19.5996826 total: 3m 30s remaining: 3m 16s 517: learn: 19.5948010 total: 3m 30s remaining: 3m 15s 518: learn: 19.5893163 total: 3m 30s remaining: 3m 15s 519: learn: 19.5823501 total: 3m 31s remaining: 3m 15s 520: learn: 19.5822679 total: 3m 31s remaining: 3m 14s 521: learn: 19.5746295 total: 3m 32s remaining: 3m 14s 522: learn: 19.5737667 total: 3m 32s remaining: 3m 13s 523: learn: 19.5493498 total: 3m 33s remaining: 3m 13s 524: learn: 19.5331516 total: 3m 33s remaining: 3m 13s 525: learn: 19.5331194 total: 3m 33s remaining: 3m 12s 526: learn: 19.5253586 total: 3m 34s remaining: 3m 12s 527: learn: 19.5141273 total: 3m 34s remaining: 3m 12s 528: learn: 19.5131151 total: 3m 35s remaining: 3m 11s 529: learn: 19.5077891 total: 3m 35s remaining: 3m 11s 530: learn: 19.5077615 total: 3m 36s remaining: 3m 11s 531: learn: 19.4977609 total: 3m 36s remaining: 3m 10s 532: learn: 19.4974333 total: 3m 37s remaining: 3m 10s 533: learn: 19.4971684 total: 3m 37s remaining: 3m 9s 534: learn: 19.4946153 total: 3m 38s remaining: 3m 9s 535: learn: 19.4945877 total: 3m 38s remaining: 3m 9s 536: learn: 19.4921575 total: 3m 38s remaining: 3m 8s 537: learn: 19.4865409 total: 3m 39s remaining: 3m 8s 538: learn: 19.4827145 total: 3m 39s remaining: 3m 7s 539: learn: 19.4777600 total: 3m 40s remaining: 3m 7s 540: learn: 19.4775911 total: 3m 40s remaining: 3m 7s 541: learn: 19.4730516 total: 3m 40s remaining: 3m 6s 542: learn: 19.4694955 total: 3m 41s remaining: 3m 6s 543: learn: 19.4634604 total: 3m 41s remaining: 3m 5s 544: learn: 19.4567713 total: 3m 42s remaining: 3m 5s 545: learn: 19.4554902 total: 3m 42s remaining: 3m 4s 546: learn: 19.4527805 total: 3m 42s remaining: 3m 4s 547: learn: 19.4527624 total: 3m 43s remaining: 3m 4s 548: learn: 19.4402256 total: 3m 43s remaining: 3m 3s 549: learn: 19.4346965 total: 3m 44s remaining: 3m 3s 550: learn: 19.4342166 total: 3m 44s remaining: 3m 2s 551: learn: 19.4323768 total: 3m 44s remaining: 3m 2s 552: learn: 19.4277640 total: 3m 45s remaining: 3m 2s 553: learn: 19.4249744 total: 3m 45s remaining: 3m 1s 554: learn: 19.4226938 total: 3m 46s remaining: 3m 1s 555: learn: 19.4160575 total: 3m 46s remaining: 3m 556: learn: 19.4111862 total: 3m 46s remaining: 3m 557: learn: 19.4085311 total: 3m 47s remaining: 3m 558: learn: 19.4066554 total: 3m 47s remaining: 2m 59s 559: learn: 19.4009236 total: 3m 48s remaining: 2m 59s 560: learn: 19.3999891 total: 3m 48s remaining: 2m 58s 561: learn: 19.3994093 total: 3m 48s remaining: 2m 58s 562: learn: 19.3916653 total: 3m 49s remaining: 2m 57s 563: learn: 19.3909186 total: 3m 49s remaining: 2m 57s 564: learn: 19.3870580 total: 3m 50s remaining: 2m 57s 565: learn: 19.3839746 total: 3m 50s remaining: 2m 56s 566: learn: 19.3817972 total: 3m 50s remaining: 2m 56s 567: learn: 19.3787845 total: 3m 51s remaining: 2m 55s 568: learn: 19.3786212 total: 3m 51s remaining: 2m 55s 569: learn: 19.3784902 total: 3m 52s remaining: 2m 55s 570: learn: 19.3732546 total: 3m 52s remaining: 2m 54s 571: learn: 19.3712934 total: 3m 52s remaining: 2m 54s 572: learn: 19.3702937 total: 3m 53s remaining: 2m 53s 573: learn: 19.3690534 total: 3m 53s remaining: 2m 53s 574: learn: 19.3654344 total: 3m 54s remaining: 2m 53s 575: learn: 19.3654176 total: 3m 54s remaining: 2m 52s 576: learn: 19.3615172 total: 3m 55s remaining: 2m 52s 577: learn: 19.3543371 total: 3m 55s remaining: 2m 51s 578: learn: 19.3490690 total: 3m 55s remaining: 2m 51s 579: learn: 19.3402412 total: 3m 56s remaining: 2m 51s 580: learn: 19.3319973 total: 3m 56s remaining: 2m 50s 581: learn: 19.3168493 total: 3m 57s remaining: 2m 50s 582: learn: 19.3064958 total: 3m 57s remaining: 2m 50s 583: learn: 19.3003262 total: 3m 58s remaining: 2m 49s 584: learn: 19.2904500 total: 3m 58s remaining: 2m 49s 585: learn: 19.2892750 total: 3m 59s remaining: 2m 48s 586: learn: 19.2810040 total: 3m 59s remaining: 2m 48s 587: learn: 19.2681543 total: 3m 59s remaining: 2m 48s 588: learn: 19.2594609 total: 4m remaining: 2m 47s 589: learn: 19.2555221 total: 4m remaining: 2m 47s 590: learn: 19.2542347 total: 4m 1s remaining: 2m 46s 591: learn: 19.2487238 total: 4m 1s remaining: 2m 46s 592: learn: 19.2465006 total: 4m 1s remaining: 2m 46s 593: learn: 19.2442517 total: 4m 2s remaining: 2m 45s 594: learn: 19.2373707 total: 4m 2s remaining: 2m 45s 595: learn: 19.2333528 total: 4m 3s remaining: 2m 44s 596: learn: 19.2242552 total: 4m 3s remaining: 2m 44s 597: learn: 19.2106914 total: 4m 4s remaining: 2m 44s 598: learn: 19.2048892 total: 4m 4s remaining: 2m 43s 599: learn: 19.2037627 total: 4m 5s remaining: 2m 43s 600: learn: 19.2017201 total: 4m 5s remaining: 2m 42s 601: learn: 19.1964880 total: 4m 5s remaining: 2m 42s 602: learn: 19.1878514 total: 4m 6s remaining: 2m 42s 603: learn: 19.1857984 total: 4m 6s remaining: 2m 41s 604: learn: 19.1674205 total: 4m 7s remaining: 2m 41s 605: learn: 19.1609108 total: 4m 7s remaining: 2m 40s 606: learn: 19.1589550 total: 4m 8s remaining: 2m 40s 607: learn: 19.1581875 total: 4m 8s remaining: 2m 40s 608: learn: 19.1549741 total: 4m 8s remaining: 2m 39s 609: learn: 19.1544684 total: 4m 9s remaining: 2m 39s 610: learn: 19.1495067 total: 4m 9s remaining: 2m 38s 611: learn: 19.1479521 total: 4m 10s remaining: 2m 38s 612: learn: 19.1442874 total: 4m 10s remaining: 2m 38s 613: learn: 19.1294303 total: 4m 10s remaining: 2m 37s 614: learn: 19.1191215 total: 4m 11s remaining: 2m 37s 615: learn: 19.1173032 total: 4m 11s remaining: 2m 36s 616: learn: 19.1151625 total: 4m 12s remaining: 2m 36s 617: learn: 19.1083085 total: 4m 12s remaining: 2m 36s 618: learn: 19.0943644 total: 4m 13s remaining: 2m 35s 619: learn: 19.0929570 total: 4m 13s remaining: 2m 35s 620: learn: 19.0861852 total: 4m 13s remaining: 2m 34s 621: learn: 19.0853468 total: 4m 14s remaining: 2m 34s 622: learn: 19.0812472 total: 4m 14s remaining: 2m 34s 623: learn: 19.0531915 total: 4m 15s remaining: 2m 33s 624: learn: 19.0418279 total: 4m 15s remaining: 2m 33s 625: learn: 19.0394624 total: 4m 15s remaining: 2m 32s 626: learn: 19.0359872 total: 4m 16s remaining: 2m 32s 627: learn: 19.0286995 total: 4m 16s remaining: 2m 32s 628: learn: 19.0220327 total: 4m 17s remaining: 2m 31s 629: learn: 19.0212648 total: 4m 17s remaining: 2m 31s 630: learn: 19.0197307 total: 4m 18s remaining: 2m 30s 631: learn: 19.0157114 total: 4m 18s remaining: 2m 30s 632: learn: 19.0106715 total: 4m 18s remaining: 2m 30s 633: learn: 19.0103449 total: 4m 19s remaining: 2m 29s 634: learn: 19.0051583 total: 4m 19s remaining: 2m 29s 635: learn: 19.0033898 total: 4m 19s remaining: 2m 28s 636: learn: 18.9906497 total: 4m 20s remaining: 2m 28s 637: learn: 18.9889222 total: 4m 20s remaining: 2m 28s 638: learn: 18.9860068 total: 4m 21s remaining: 2m 27s 639: learn: 18.9812598 total: 4m 21s remaining: 2m 27s 640: learn: 18.9723380 total: 4m 22s remaining: 2m 26s 641: learn: 18.9700109 total: 4m 22s remaining: 2m 26s 642: learn: 18.9659261 total: 4m 22s remaining: 2m 25s 643: learn: 18.9659253 total: 4m 23s remaining: 2m 25s 644: learn: 18.9627562 total: 4m 23s remaining: 2m 25s 645: learn: 18.9595341 total: 4m 24s remaining: 2m 24s 646: learn: 18.9505143 total: 4m 24s remaining: 2m 24s 647: learn: 18.9456990 total: 4m 24s remaining: 2m 23s 648: learn: 18.9397654 total: 4m 25s remaining: 2m 23s 649: learn: 18.9391771 total: 4m 25s remaining: 2m 23s 650: learn: 18.9385090 total: 4m 25s remaining: 2m 22s 651: learn: 18.9350580 total: 4m 26s remaining: 2m 22s 652: learn: 18.9287725 total: 4m 26s remaining: 2m 21s 653: learn: 18.9273496 total: 4m 27s remaining: 2m 21s 654: learn: 18.9250325 total: 4m 27s remaining: 2m 20s 655: learn: 18.9142647 total: 4m 27s remaining: 2m 20s 656: learn: 18.8974858 total: 4m 28s remaining: 2m 20s 657: learn: 18.8960973 total: 4m 28s remaining: 2m 19s 658: learn: 18.8927113 total: 4m 29s remaining: 2m 19s 659: learn: 18.8903306 total: 4m 29s remaining: 2m 18s 660: learn: 18.8891331 total: 4m 30s remaining: 2m 18s 661: learn: 18.8775029 total: 4m 30s remaining: 2m 18s 662: learn: 18.8673254 total: 4m 30s remaining: 2m 17s 663: learn: 18.8672304 total: 4m 31s remaining: 2m 17s 664: learn: 18.8670170 total: 4m 31s remaining: 2m 16s 665: learn: 18.8622432 total: 4m 31s remaining: 2m 16s 666: learn: 18.8600927 total: 4m 32s remaining: 2m 15s 667: learn: 18.8589378 total: 4m 32s remaining: 2m 15s 668: learn: 18.8557385 total: 4m 32s remaining: 2m 15s 669: learn: 18.8529518 total: 4m 33s remaining: 2m 14s 670: learn: 18.8195596 total: 4m 33s remaining: 2m 14s 671: learn: 18.8160823 total: 4m 34s remaining: 2m 13s 672: learn: 18.8118896 total: 4m 34s remaining: 2m 13s 673: learn: 18.8095102 total: 4m 34s remaining: 2m 12s 674: learn: 18.7995551 total: 4m 35s remaining: 2m 12s 675: learn: 18.7926271 total: 4m 35s remaining: 2m 12s 676: learn: 18.7877059 total: 4m 36s remaining: 2m 11s 677: learn: 18.7865284 total: 4m 36s remaining: 2m 11s 678: learn: 18.7810344 total: 4m 36s remaining: 2m 10s 679: learn: 18.7724689 total: 4m 37s remaining: 2m 10s 680: learn: 18.7722842 total: 4m 37s remaining: 2m 10s 681: learn: 18.7668981 total: 4m 37s remaining: 2m 9s 682: learn: 18.7668329 total: 4m 38s remaining: 2m 9s 683: learn: 18.7590444 total: 4m 38s remaining: 2m 8s 684: learn: 18.7560313 total: 4m 39s remaining: 2m 8s 685: learn: 18.7475425 total: 4m 39s remaining: 2m 7s 686: learn: 18.7465439 total: 4m 39s remaining: 2m 7s 687: learn: 18.7430805 total: 4m 40s remaining: 2m 7s 688: learn: 18.7297214 total: 4m 40s remaining: 2m 6s 689: learn: 18.7251161 total: 4m 41s remaining: 2m 6s 690: learn: 18.7186370 total: 4m 41s remaining: 2m 5s 691: learn: 18.7137987 total: 4m 41s remaining: 2m 5s 692: learn: 18.7094481 total: 4m 42s remaining: 2m 5s 693: learn: 18.7094468 total: 4m 42s remaining: 2m 4s 694: learn: 18.6984832 total: 4m 42s remaining: 2m 4s 695: learn: 18.6984820 total: 4m 43s remaining: 2m 3s 696: learn: 18.6950050 total: 4m 43s remaining: 2m 3s 697: learn: 18.6898949 total: 4m 44s remaining: 2m 2s 698: learn: 18.6844497 total: 4m 44s remaining: 2m 2s 699: learn: 18.6818053 total: 4m 44s remaining: 2m 2s 700: learn: 18.6792691 total: 4m 45s remaining: 2m 1s 701: learn: 18.6624308 total: 4m 45s remaining: 2m 1s 702: learn: 18.6602364 total: 4m 45s remaining: 2m 703: learn: 18.6543374 total: 4m 46s remaining: 2m 704: learn: 18.6493318 total: 4m 46s remaining: 1m 59s 705: learn: 18.6417329 total: 4m 47s remaining: 1m 59s 706: learn: 18.6344113 total: 4m 47s remaining: 1m 59s 707: learn: 18.6311694 total: 4m 47s remaining: 1m 58s 708: learn: 18.6286134 total: 4m 48s remaining: 1m 58s 709: learn: 18.6263733 total: 4m 48s remaining: 1m 57s 710: learn: 18.6189948 total: 4m 48s remaining: 1m 57s 711: learn: 18.6175325 total: 4m 49s remaining: 1m 57s 712: learn: 18.6146914 total: 4m 49s remaining: 1m 56s 713: learn: 18.6070692 total: 4m 50s remaining: 1m 56s 714: learn: 18.6021655 total: 4m 50s remaining: 1m 55s 715: learn: 18.5961335 total: 4m 50s remaining: 1m 55s 716: learn: 18.5959633 total: 4m 51s remaining: 1m 54s 717: learn: 18.5942007 total: 4m 51s remaining: 1m 54s 718: learn: 18.5906828 total: 4m 52s remaining: 1m 54s 719: learn: 18.5896072 total: 4m 52s remaining: 1m 53s 720: learn: 18.5888060 total: 4m 52s remaining: 1m 53s 721: learn: 18.5876242 total: 4m 53s remaining: 1m 52s 722: learn: 18.5867614 total: 4m 53s remaining: 1m 52s 723: learn: 18.5816359 total: 4m 54s remaining: 1m 52s 724: learn: 18.5808036 total: 4m 54s remaining: 1m 51s 725: learn: 18.5787143 total: 4m 54s remaining: 1m 51s 726: learn: 18.5731384 total: 4m 55s remaining: 1m 50s 727: learn: 18.5684553 total: 4m 55s remaining: 1m 50s 728: learn: 18.5620002 total: 4m 56s remaining: 1m 50s 729: learn: 18.5592303 total: 4m 56s remaining: 1m 49s 730: learn: 18.5582778 total: 4m 56s remaining: 1m 49s 731: learn: 18.5484938 total: 4m 57s remaining: 1m 48s 732: learn: 18.5476295 total: 4m 57s remaining: 1m 48s 733: learn: 18.5379602 total: 4m 58s remaining: 1m 48s 734: learn: 18.5370244 total: 4m 58s remaining: 1m 47s 735: learn: 18.5363160 total: 4m 58s remaining: 1m 47s 736: learn: 18.5337508 total: 4m 59s remaining: 1m 46s 737: learn: 18.5330612 total: 4m 59s remaining: 1m 46s 738: learn: 18.5261400 total: 5m remaining: 1m 45s 739: learn: 18.5225358 total: 5m remaining: 1m 45s 740: learn: 18.5219627 total: 5m remaining: 1m 45s 741: learn: 18.5179011 total: 5m 1s remaining: 1m 44s 742: learn: 18.5047575 total: 5m 1s remaining: 1m 44s 743: learn: 18.5029936 total: 5m 1s remaining: 1m 43s 744: learn: 18.5011830 total: 5m 2s remaining: 1m 43s 745: learn: 18.5007644 total: 5m 2s remaining: 1m 43s 746: learn: 18.4969535 total: 5m 3s remaining: 1m 42s 747: learn: 18.4850511 total: 5m 3s remaining: 1m 42s 748: learn: 18.4823046 total: 5m 3s remaining: 1m 41s 749: learn: 18.4816655 total: 5m 4s remaining: 1m 41s 750: learn: 18.4784166 total: 5m 4s remaining: 1m 41s 751: learn: 18.4759609 total: 5m 5s remaining: 1m 40s 752: learn: 18.4754251 total: 5m 5s remaining: 1m 40s 753: learn: 18.4731784 total: 5m 5s remaining: 1m 39s 754: learn: 18.4726545 total: 5m 6s remaining: 1m 39s 755: learn: 18.4701925 total: 5m 6s remaining: 1m 38s 756: learn: 18.4622218 total: 5m 6s remaining: 1m 38s 757: learn: 18.4611381 total: 5m 7s remaining: 1m 38s 758: learn: 18.4610279 total: 5m 7s remaining: 1m 37s 759: learn: 18.4564583 total: 5m 8s remaining: 1m 37s 760: learn: 18.4442539 total: 5m 8s remaining: 1m 36s 761: learn: 18.4391964 total: 5m 8s remaining: 1m 36s 762: learn: 18.4309098 total: 5m 9s remaining: 1m 36s 763: learn: 18.4263199 total: 5m 9s remaining: 1m 35s 764: learn: 18.4254016 total: 5m 10s remaining: 1m 35s 765: learn: 18.4038961 total: 5m 10s remaining: 1m 34s 766: learn: 18.3972924 total: 5m 10s remaining: 1m 34s 767: learn: 18.3962660 total: 5m 11s remaining: 1m 34s 768: learn: 18.3847346 total: 5m 11s remaining: 1m 33s 769: learn: 18.3697992 total: 5m 12s remaining: 1m 33s 770: learn: 18.3673326 total: 5m 12s remaining: 1m 32s 771: learn: 18.3584073 total: 5m 13s remaining: 1m 32s 772: learn: 18.3558821 total: 5m 13s remaining: 1m 32s 773: learn: 18.3547031 total: 5m 13s remaining: 1m 31s 774: learn: 18.3531089 total: 5m 14s remaining: 1m 31s 775: learn: 18.3518404 total: 5m 14s remaining: 1m 30s 776: learn: 18.3488056 total: 5m 14s remaining: 1m 30s 777: learn: 18.3474509 total: 5m 15s remaining: 1m 29s 778: learn: 18.3418036 total: 5m 15s remaining: 1m 29s 779: learn: 18.3407466 total: 5m 16s remaining: 1m 29s 780: learn: 18.3395556 total: 5m 16s remaining: 1m 28s 781: learn: 18.3380022 total: 5m 16s remaining: 1m 28s 782: learn: 18.3359755 total: 5m 17s remaining: 1m 27s 783: learn: 18.3276076 total: 5m 17s remaining: 1m 27s 784: learn: 18.3252568 total: 5m 18s remaining: 1m 27s 785: learn: 18.3219793 total: 5m 18s remaining: 1m 26s 786: learn: 18.3197668 total: 5m 18s remaining: 1m 26s 787: learn: 18.3180245 total: 5m 19s remaining: 1m 25s 788: learn: 18.3167874 total: 5m 19s remaining: 1m 25s 789: learn: 18.3159257 total: 5m 20s remaining: 1m 25s 790: learn: 18.3076412 total: 5m 20s remaining: 1m 24s 791: learn: 18.3068163 total: 5m 20s remaining: 1m 24s 792: learn: 18.2936001 total: 5m 21s remaining: 1m 23s 793: learn: 18.2796844 total: 5m 21s remaining: 1m 23s 794: learn: 18.2785824 total: 5m 21s remaining: 1m 23s 795: learn: 18.2770695 total: 5m 22s remaining: 1m 22s 796: learn: 18.2760435 total: 5m 22s remaining: 1m 22s 797: learn: 18.2727108 total: 5m 23s remaining: 1m 21s 798: learn: 18.2710321 total: 5m 23s remaining: 1m 21s 799: learn: 18.2705636 total: 5m 23s remaining: 1m 20s 800: learn: 18.2688425 total: 5m 24s remaining: 1m 20s 801: learn: 18.2647590 total: 5m 24s remaining: 1m 20s 802: learn: 18.2641228 total: 5m 25s remaining: 1m 19s 803: learn: 18.2632807 total: 5m 25s remaining: 1m 19s 804: learn: 18.2622981 total: 5m 25s remaining: 1m 18s 805: learn: 18.2608318 total: 5m 26s remaining: 1m 18s 806: learn: 18.2579290 total: 5m 26s remaining: 1m 18s 807: learn: 18.2571818 total: 5m 27s remaining: 1m 17s 808: learn: 18.2565919 total: 5m 27s remaining: 1m 17s 809: learn: 18.2516194 total: 5m 27s remaining: 1m 16s 810: learn: 18.2495051 total: 5m 28s remaining: 1m 16s 811: learn: 18.2411633 total: 5m 28s remaining: 1m 16s 812: learn: 18.2370452 total: 5m 29s remaining: 1m 15s 813: learn: 18.2327284 total: 5m 29s remaining: 1m 15s 814: learn: 18.2309726 total: 5m 29s remaining: 1m 14s 815: learn: 18.2299879 total: 5m 30s remaining: 1m 14s 816: learn: 18.2294127 total: 5m 30s remaining: 1m 14s 817: learn: 18.2259516 total: 5m 31s remaining: 1m 13s 818: learn: 18.2066354 total: 5m 31s remaining: 1m 13s 819: learn: 18.2042684 total: 5m 31s remaining: 1m 12s 820: learn: 18.2012420 total: 5m 32s remaining: 1m 12s 821: learn: 18.1943036 total: 5m 32s remaining: 1m 12s 822: learn: 18.1942404 total: 5m 33s remaining: 1m 11s 823: learn: 18.1906316 total: 5m 33s remaining: 1m 11s 824: learn: 18.1894128 total: 5m 33s remaining: 1m 10s 825: learn: 18.1884689 total: 5m 34s remaining: 1m 10s 826: learn: 18.1778205 total: 5m 34s remaining: 1m 10s 827: learn: 18.1736319 total: 5m 35s remaining: 1m 9s 828: learn: 18.1734134 total: 5m 35s remaining: 1m 9s 829: learn: 18.1704374 total: 5m 35s remaining: 1m 8s 830: learn: 18.1692078 total: 5m 36s remaining: 1m 8s 831: learn: 18.1662468 total: 5m 36s remaining: 1m 7s 832: learn: 18.1609406 total: 5m 37s remaining: 1m 7s 833: learn: 18.1607981 total: 5m 37s remaining: 1m 7s 834: learn: 18.1394563 total: 5m 37s remaining: 1m 6s 835: learn: 18.1329315 total: 5m 38s remaining: 1m 6s 836: learn: 18.1292822 total: 5m 38s remaining: 1m 5s 837: learn: 18.1231097 total: 5m 38s remaining: 1m 5s 838: learn: 18.1226114 total: 5m 39s remaining: 1m 5s 839: learn: 18.1180640 total: 5m 39s remaining: 1m 4s 840: learn: 18.1175282 total: 5m 40s remaining: 1m 4s 841: learn: 18.1141392 total: 5m 40s remaining: 1m 3s 842: learn: 18.1120931 total: 5m 40s remaining: 1m 3s 843: learn: 18.1024538 total: 5m 41s remaining: 1m 3s 844: learn: 18.1007412 total: 5m 41s remaining: 1m 2s 845: learn: 18.1004167 total: 5m 42s remaining: 1m 2s 846: learn: 18.0890868 total: 5m 42s remaining: 1m 1s 847: learn: 18.0875512 total: 5m 42s remaining: 1m 1s 848: learn: 18.0788778 total: 5m 43s remaining: 1m 1s 849: learn: 18.0725789 total: 5m 43s remaining: 1m 850: learn: 18.0716034 total: 5m 44s remaining: 1m 851: learn: 18.0710676 total: 5m 44s remaining: 59.9s 852: learn: 18.0689879 total: 5m 45s remaining: 59.5s 853: learn: 18.0680257 total: 5m 45s remaining: 59.1s 854: learn: 18.0666869 total: 5m 45s remaining: 58.7s 855: learn: 18.0653872 total: 5m 46s remaining: 58.2s 856: learn: 18.0563583 total: 5m 46s remaining: 57.8s 857: learn: 18.0553548 total: 5m 47s remaining: 57.4s 858: learn: 18.0526909 total: 5m 47s remaining: 57s 859: learn: 18.0394855 total: 5m 47s remaining: 56.6s 860: learn: 18.0378398 total: 5m 48s remaining: 56.2s 861: learn: 18.0346769 total: 5m 48s remaining: 55.8s 862: learn: 18.0328518 total: 5m 49s remaining: 55.4s 863: learn: 18.0315766 total: 5m 49s remaining: 55s 864: learn: 18.0311528 total: 5m 49s remaining: 54.6s 865: learn: 18.0268530 total: 5m 50s remaining: 54.2s 866: learn: 18.0237832 total: 5m 50s remaining: 53.8s 867: learn: 18.0227374 total: 5m 51s remaining: 53.4s 868: learn: 18.0219677 total: 5m 51s remaining: 53s 869: learn: 18.0113964 total: 5m 51s remaining: 52.6s 870: learn: 18.0094218 total: 5m 52s remaining: 52.2s 871: learn: 18.0086332 total: 5m 52s remaining: 51.8s 872: learn: 18.0053578 total: 5m 53s remaining: 51.4s 873: learn: 18.0045638 total: 5m 53s remaining: 50.9s 874: learn: 18.0044606 total: 5m 53s remaining: 50.5s 875: learn: 18.0029449 total: 5m 54s remaining: 50.1s 876: learn: 18.0027901 total: 5m 54s remaining: 49.7s 877: learn: 17.9996863 total: 5m 54s remaining: 49.3s 878: learn: 17.9982643 total: 5m 55s remaining: 48.9s 879: learn: 17.9970783 total: 5m 55s remaining: 48.5s 880: learn: 17.9944870 total: 5m 56s remaining: 48.1s 881: learn: 17.9938737 total: 5m 56s remaining: 47.7s 882: learn: 17.9932421 total: 5m 57s remaining: 47.3s 883: learn: 17.9925276 total: 5m 57s remaining: 46.9s 884: learn: 17.9920922 total: 5m 57s remaining: 46.5s 885: learn: 17.9920847 total: 5m 58s remaining: 46.1s 886: learn: 17.9917835 total: 5m 58s remaining: 45.7s 887: learn: 17.9905055 total: 5m 59s remaining: 45.3s 888: learn: 17.9897115 total: 5m 59s remaining: 44.9s 889: learn: 17.9874312 total: 5m 59s remaining: 44.5s 890: learn: 17.9866969 total: 6m remaining: 44.1s 891: learn: 17.9802328 total: 6m remaining: 43.7s 892: learn: 17.9768436 total: 6m remaining: 43.3s 893: learn: 17.9761631 total: 6m 1s remaining: 42.8s 894: learn: 17.9738052 total: 6m 1s remaining: 42.4s 895: learn: 17.9735286 total: 6m 2s remaining: 42s 896: learn: 17.9731124 total: 6m 2s remaining: 41.6s 897: learn: 17.9717256 total: 6m 2s remaining: 41.2s 898: learn: 17.9644389 total: 6m 3s remaining: 40.8s 899: learn: 17.9578194 total: 6m 3s remaining: 40.4s 900: learn: 17.9569545 total: 6m 4s remaining: 40s 901: learn: 17.9544222 total: 6m 4s remaining: 39.6s 902: learn: 17.9516295 total: 6m 4s remaining: 39.2s 903: learn: 17.9485373 total: 6m 5s remaining: 38.8s 904: learn: 17.9465701 total: 6m 5s remaining: 38.4s 905: learn: 17.9433465 total: 6m 6s remaining: 38s 906: learn: 17.9332212 total: 6m 6s remaining: 37.6s 907: learn: 17.9236631 total: 6m 6s remaining: 37.2s 908: learn: 17.9165058 total: 6m 7s remaining: 36.8s 909: learn: 17.9141353 total: 6m 7s remaining: 36.4s 910: learn: 17.9121656 total: 6m 8s remaining: 36s 911: learn: 17.9066900 total: 6m 8s remaining: 35.6s 912: learn: 17.8978376 total: 6m 8s remaining: 35.2s 913: learn: 17.8947106 total: 6m 9s remaining: 34.8s 914: learn: 17.8925449 total: 6m 9s remaining: 34.3s 915: learn: 17.8869343 total: 6m 10s remaining: 33.9s 916: learn: 17.8848926 total: 6m 10s remaining: 33.5s 917: learn: 17.8840528 total: 6m 10s remaining: 33.1s 918: learn: 17.8825361 total: 6m 11s remaining: 32.7s 919: learn: 17.8791601 total: 6m 11s remaining: 32.3s 920: learn: 17.8763267 total: 6m 12s remaining: 31.9s 921: learn: 17.8752407 total: 6m 12s remaining: 31.5s 922: learn: 17.8749161 total: 6m 12s remaining: 31.1s 923: learn: 17.8722891 total: 6m 13s remaining: 30.7s 924: learn: 17.8673611 total: 6m 13s remaining: 30.3s 925: learn: 17.8654082 total: 6m 13s remaining: 29.9s 926: learn: 17.8613767 total: 6m 14s remaining: 29.5s 927: learn: 17.8592576 total: 6m 14s remaining: 29.1s 928: learn: 17.8583836 total: 6m 15s remaining: 28.7s 929: learn: 17.8432779 total: 6m 15s remaining: 28.3s 930: learn: 17.8384619 total: 6m 15s remaining: 27.9s 931: learn: 17.8363800 total: 6m 16s remaining: 27.4s 932: learn: 17.8331894 total: 6m 16s remaining: 27s 933: learn: 17.8278576 total: 6m 16s remaining: 26.6s 934: learn: 17.8244080 total: 6m 17s remaining: 26.2s 935: learn: 17.8191569 total: 6m 17s remaining: 25.8s 936: learn: 17.8176770 total: 6m 18s remaining: 25.4s 937: learn: 17.8175093 total: 6m 18s remaining: 25s 938: learn: 17.8156661 total: 6m 18s remaining: 24.6s 939: learn: 17.8147569 total: 6m 19s remaining: 24.2s 940: learn: 17.8127135 total: 6m 19s remaining: 23.8s 941: learn: 17.8047875 total: 6m 20s remaining: 23.4s 942: learn: 17.8013534 total: 6m 20s remaining: 23s 943: learn: 17.8002952 total: 6m 20s remaining: 22.6s 944: learn: 17.7933684 total: 6m 21s remaining: 22.2s 945: learn: 17.7925565 total: 6m 21s remaining: 21.8s 946: learn: 17.7897267 total: 6m 22s remaining: 21.4s 947: learn: 17.7881392 total: 6m 22s remaining: 21s 948: learn: 17.7865810 total: 6m 22s remaining: 20.6s 949: learn: 17.7826133 total: 6m 23s remaining: 20.2s 950: learn: 17.7822981 total: 6m 23s remaining: 19.8s 951: learn: 17.7746169 total: 6m 24s remaining: 19.4s 952: learn: 17.7716954 total: 6m 24s remaining: 19s 953: learn: 17.7632942 total: 6m 24s remaining: 18.6s 954: learn: 17.7629191 total: 6m 25s remaining: 18.2s 955: learn: 17.7566433 total: 6m 25s remaining: 17.7s 956: learn: 17.7507967 total: 6m 26s remaining: 17.3s 957: learn: 17.7501615 total: 6m 26s remaining: 16.9s 958: learn: 17.7384997 total: 6m 26s remaining: 16.5s 959: learn: 17.7356531 total: 6m 27s remaining: 16.1s 960: learn: 17.7298015 total: 6m 27s remaining: 15.7s 961: learn: 17.7292051 total: 6m 28s remaining: 15.3s 962: learn: 17.7167874 total: 6m 28s remaining: 14.9s 963: learn: 17.7119552 total: 6m 28s remaining: 14.5s 964: learn: 17.7034379 total: 6m 29s remaining: 14.1s 965: learn: 17.6977633 total: 6m 29s remaining: 13.7s 966: learn: 17.6972394 total: 6m 29s remaining: 13.3s 967: learn: 17.6958169 total: 6m 30s remaining: 12.9s 968: learn: 17.6940753 total: 6m 30s remaining: 12.5s 969: learn: 17.6936256 total: 6m 31s remaining: 12.1s 970: learn: 17.6928848 total: 6m 31s remaining: 11.7s 971: learn: 17.6925482 total: 6m 31s remaining: 11.3s 972: learn: 17.6906856 total: 6m 32s remaining: 10.9s 973: learn: 17.6849220 total: 6m 32s remaining: 10.5s 974: learn: 17.6811028 total: 6m 33s remaining: 10.1s 975: learn: 17.6749462 total: 6m 33s remaining: 9.68s 976: learn: 17.6740867 total: 6m 33s remaining: 9.27s 977: learn: 17.6729584 total: 6m 34s remaining: 8.87s 978: learn: 17.6677012 total: 6m 34s remaining: 8.46s 979: learn: 17.6669882 total: 6m 34s remaining: 8.06s 980: learn: 17.6664851 total: 6m 35s remaining: 7.66s 981: learn: 17.6649882 total: 6m 35s remaining: 7.25s 982: learn: 17.6640244 total: 6m 36s remaining: 6.85s 983: learn: 17.6620184 total: 6m 36s remaining: 6.45s 984: learn: 17.6611420 total: 6m 36s remaining: 6.04s 985: learn: 17.6610167 total: 6m 37s remaining: 5.64s 986: learn: 17.6585368 total: 6m 37s remaining: 5.24s 987: learn: 17.6577112 total: 6m 38s remaining: 4.83s 988: learn: 17.6574431 total: 6m 38s remaining: 4.43s 989: learn: 17.6534576 total: 6m 38s remaining: 4.03s 990: learn: 17.6527593 total: 6m 39s remaining: 3.63s 991: learn: 17.6303109 total: 6m 39s remaining: 3.22s 992: learn: 17.6282744 total: 6m 40s remaining: 2.82s 993: learn: 17.6257135 total: 6m 40s remaining: 2.42s 994: learn: 17.6198157 total: 6m 40s remaining: 2.02s 995: learn: 17.6185890 total: 6m 41s remaining: 1.61s 996: learn: 17.6171366 total: 6m 41s remaining: 1.21s 997: learn: 17.6082536 total: 6m 42s remaining: 806ms 998: learn: 17.6070686 total: 6m 42s remaining: 403ms 999: learn: 17.6044398 total: 6m 43s remaining: 0us
<catboost.core.CatBoostRegressor at 0x26a060a6fd0>
#Получение ответов модели на тестовой выборке в локальном тестировании
y_pred = model.predict(pool_test)
#На локальном тестировании модель выдаёт такой результат
print("Значение метрики R2 на test: ", r2_score(y_test, y_pred))
Значение метрики R2 на test: 0.8682238751045731
#Формируем sample_solution. В обучении используется весь train, ответы получаем на test
pool_train_solution = Pool(X, y, cat_features = ['PATIENT_SEX', 'MKB_CODE', 'ADRES', 'VISIT_MONTH_YEAR', 'AGE_CATEGORY'])
pool_test_solution = Pool(test, cat_features = ['PATIENT_SEX', 'MKB_CODE', 'ADRES', 'VISIT_MONTH_YEAR', 'AGE_CATEGORY'])
#model_solution = CatBoostRegressor(task_type='GPU')
model_solution = CatBoostRegressor()
model_solution.fit(pool_train_solution)
Learning rate set to 0.138251 0: learn: 56.2453021 total: 1.56s remaining: 25m 55s 1: learn: 52.9985497 total: 2.62s remaining: 21m 49s 2: learn: 50.3934034 total: 3.21s remaining: 17m 46s 3: learn: 48.2486162 total: 3.68s remaining: 15m 16s 4: learn: 46.4699565 total: 4.85s remaining: 16m 5s 5: learn: 45.0898618 total: 5.34s remaining: 14m 45s 6: learn: 44.0292792 total: 5.8s remaining: 13m 43s 7: learn: 41.9382723 total: 6.7s remaining: 13m 51s 8: learn: 40.4035953 total: 7.51s remaining: 13m 46s 9: learn: 39.0002847 total: 8.3s remaining: 13m 41s 10: learn: 38.2645636 total: 8.78s remaining: 13m 9s 11: learn: 37.3417849 total: 9.28s remaining: 12m 44s 12: learn: 36.5966932 total: 9.81s remaining: 12m 24s 13: learn: 36.1133737 total: 10.5s remaining: 12m 17s 14: learn: 35.7550837 total: 11.2s remaining: 12m 14s 15: learn: 35.3324396 total: 11.8s remaining: 12m 7s 16: learn: 35.0018434 total: 12.4s remaining: 11m 54s 17: learn: 34.6405552 total: 13.5s remaining: 12m 19s 18: learn: 34.3168049 total: 14s remaining: 12m 1s 19: learn: 34.0176978 total: 14.6s remaining: 11m 55s 20: learn: 33.7883452 total: 15.8s remaining: 12m 18s 21: learn: 33.5507747 total: 16.6s remaining: 12m 19s 22: learn: 33.0925052 total: 17.2s remaining: 12m 12s 23: learn: 32.7301868 total: 18.1s remaining: 12m 16s 24: learn: 32.4242812 total: 19.3s remaining: 12m 31s 25: learn: 32.1929619 total: 19.7s remaining: 12m 19s 26: learn: 31.9310707 total: 20.2s remaining: 12m 9s 27: learn: 31.6111387 total: 21.1s remaining: 12m 13s 28: learn: 31.4374885 total: 21.6s remaining: 12m 3s 29: learn: 31.3231755 total: 22.1s remaining: 11m 53s 30: learn: 31.1426485 total: 22.6s remaining: 11m 45s 31: learn: 31.0495330 total: 23s remaining: 11m 36s 32: learn: 30.9145991 total: 23.5s remaining: 11m 29s 33: learn: 30.7895017 total: 24s remaining: 11m 21s 34: learn: 30.5821709 total: 24.5s remaining: 11m 15s 35: learn: 30.4474702 total: 25s remaining: 11m 9s 36: learn: 30.2737673 total: 25.7s remaining: 11m 8s 37: learn: 30.1960229 total: 26.1s remaining: 11m 2s 38: learn: 30.1120834 total: 26.6s remaining: 10m 55s 39: learn: 30.0495685 total: 27.1s remaining: 10m 49s 40: learn: 29.9135172 total: 27.5s remaining: 10m 44s 41: learn: 29.5944824 total: 28s remaining: 10m 38s 42: learn: 29.4941628 total: 28.5s remaining: 10m 34s 43: learn: 29.3732663 total: 29s remaining: 10m 29s 44: learn: 29.2991964 total: 29.5s remaining: 10m 25s 45: learn: 29.2036677 total: 29.9s remaining: 10m 20s 46: learn: 29.0812374 total: 30.4s remaining: 10m 16s 47: learn: 28.9886971 total: 30.9s remaining: 10m 13s 48: learn: 28.9639224 total: 31.4s remaining: 10m 8s 49: learn: 28.7176562 total: 31.8s remaining: 10m 4s 50: learn: 28.6660187 total: 32.3s remaining: 10m 51: learn: 28.5493574 total: 32.7s remaining: 9m 56s 52: learn: 28.4423079 total: 33.2s remaining: 9m 53s 53: learn: 28.3811296 total: 33.7s remaining: 9m 49s 54: learn: 28.3294028 total: 34.2s remaining: 9m 47s 55: learn: 28.2296007 total: 34.7s remaining: 9m 44s 56: learn: 28.1617139 total: 35.2s remaining: 9m 42s 57: learn: 28.1477301 total: 35.7s remaining: 9m 40s 58: learn: 28.0799982 total: 36.2s remaining: 9m 38s 59: learn: 28.0199622 total: 36.8s remaining: 9m 36s 60: learn: 28.0049050 total: 37.3s remaining: 9m 33s 61: learn: 27.7927901 total: 37.7s remaining: 9m 31s 62: learn: 27.7272527 total: 38.5s remaining: 9m 31s 63: learn: 27.6498775 total: 38.9s remaining: 9m 29s 64: learn: 27.6107756 total: 39.4s remaining: 9m 27s 65: learn: 27.6007752 total: 39.9s remaining: 9m 24s 66: learn: 27.6007378 total: 40.3s remaining: 9m 20s 67: learn: 27.4657087 total: 40.8s remaining: 9m 18s 68: learn: 27.2746277 total: 41.3s remaining: 9m 17s 69: learn: 27.2389774 total: 41.9s remaining: 9m 16s 70: learn: 27.2214138 total: 42.3s remaining: 9m 14s 71: learn: 27.0681346 total: 42.8s remaining: 9m 11s 72: learn: 26.9752506 total: 43.3s remaining: 9m 9s 73: learn: 26.9059813 total: 43.7s remaining: 9m 7s 74: learn: 26.8577717 total: 44.3s remaining: 9m 6s 75: learn: 26.8237577 total: 44.9s remaining: 9m 5s 76: learn: 26.7796441 total: 45.3s remaining: 9m 3s 77: learn: 26.7529573 total: 45.9s remaining: 9m 2s 78: learn: 26.5780745 total: 46.4s remaining: 9m 1s 79: learn: 26.5575755 total: 47s remaining: 9m 80: learn: 26.5217549 total: 47.5s remaining: 8m 59s 81: learn: 26.4799672 total: 48s remaining: 8m 57s 82: learn: 26.4590276 total: 48.5s remaining: 8m 56s 83: learn: 26.3875119 total: 49s remaining: 8m 54s 84: learn: 26.3440676 total: 49.5s remaining: 8m 52s 85: learn: 26.2490187 total: 49.9s remaining: 8m 50s 86: learn: 26.2038180 total: 50.4s remaining: 8m 49s 87: learn: 26.1651699 total: 50.9s remaining: 8m 47s 88: learn: 26.1152987 total: 51.4s remaining: 8m 46s 89: learn: 26.0675476 total: 51.9s remaining: 8m 45s 90: learn: 26.0591651 total: 52.5s remaining: 8m 43s 91: learn: 26.0591587 total: 52.8s remaining: 8m 40s 92: learn: 26.0187956 total: 53.3s remaining: 8m 39s 93: learn: 26.0068937 total: 53.7s remaining: 8m 37s 94: learn: 25.9120616 total: 54.3s remaining: 8m 37s 95: learn: 25.8504605 total: 54.9s remaining: 8m 36s 96: learn: 25.8375256 total: 55.3s remaining: 8m 35s 97: learn: 25.7862384 total: 55.8s remaining: 8m 33s 98: learn: 25.7847119 total: 56.2s remaining: 8m 31s 99: learn: 25.7808576 total: 56.7s remaining: 8m 30s 100: learn: 25.7460289 total: 57.2s remaining: 8m 28s 101: learn: 25.7413629 total: 57.6s remaining: 8m 27s 102: learn: 25.7171197 total: 58.1s remaining: 8m 25s 103: learn: 25.7002366 total: 58.6s remaining: 8m 24s 104: learn: 25.6486727 total: 59.1s remaining: 8m 23s 105: learn: 25.5676445 total: 59.6s remaining: 8m 22s 106: learn: 25.5341533 total: 1m remaining: 8m 21s 107: learn: 25.5249909 total: 1m remaining: 8m 20s 108: learn: 25.5228170 total: 1m 1s remaining: 8m 19s 109: learn: 25.4725361 total: 1m 1s remaining: 8m 18s 110: learn: 25.4036840 total: 1m 2s remaining: 8m 16s 111: learn: 25.3717426 total: 1m 2s remaining: 8m 15s 112: learn: 25.3717349 total: 1m 3s remaining: 8m 14s 113: learn: 25.2849462 total: 1m 3s remaining: 8m 13s 114: learn: 25.2678087 total: 1m 3s remaining: 8m 12s 115: learn: 25.2255925 total: 1m 4s remaining: 8m 10s 116: learn: 25.2075524 total: 1m 4s remaining: 8m 9s 117: learn: 25.1525322 total: 1m 5s remaining: 8m 8s 118: learn: 25.1022143 total: 1m 5s remaining: 8m 7s 119: learn: 25.0401651 total: 1m 6s remaining: 8m 6s 120: learn: 25.0365934 total: 1m 6s remaining: 8m 5s 121: learn: 24.9848844 total: 1m 7s remaining: 8m 4s 122: learn: 24.9825635 total: 1m 7s remaining: 8m 3s 123: learn: 24.9701898 total: 1m 8s remaining: 8m 1s 124: learn: 24.9383324 total: 1m 8s remaining: 8m 1s 125: learn: 24.9383265 total: 1m 9s remaining: 8m 126: learn: 24.8766668 total: 1m 9s remaining: 7m 59s 127: learn: 24.8766648 total: 1m 10s remaining: 7m 58s 128: learn: 24.8447520 total: 1m 10s remaining: 7m 58s 129: learn: 24.8129169 total: 1m 11s remaining: 7m 57s 130: learn: 24.7539128 total: 1m 11s remaining: 7m 56s 131: learn: 24.7534546 total: 1m 12s remaining: 7m 55s 132: learn: 24.6869352 total: 1m 12s remaining: 7m 53s 133: learn: 24.6296049 total: 1m 13s remaining: 7m 52s 134: learn: 24.6177468 total: 1m 13s remaining: 7m 51s 135: learn: 24.5782594 total: 1m 14s remaining: 7m 50s 136: learn: 24.5305248 total: 1m 14s remaining: 7m 49s 137: learn: 24.5075483 total: 1m 15s remaining: 7m 48s 138: learn: 24.4720751 total: 1m 15s remaining: 7m 47s 139: learn: 24.4607332 total: 1m 16s remaining: 7m 47s 140: learn: 24.4376497 total: 1m 16s remaining: 7m 46s 141: learn: 24.4349067 total: 1m 17s remaining: 7m 45s 142: learn: 24.3990406 total: 1m 17s remaining: 7m 44s 143: learn: 24.3953010 total: 1m 18s remaining: 7m 43s 144: learn: 24.3924832 total: 1m 18s remaining: 7m 42s 145: learn: 24.3902013 total: 1m 19s remaining: 7m 42s 146: learn: 24.3876503 total: 1m 19s remaining: 7m 41s 147: learn: 24.3348685 total: 1m 20s remaining: 7m 40s 148: learn: 24.3059781 total: 1m 20s remaining: 7m 39s 149: learn: 24.2534842 total: 1m 20s remaining: 7m 38s 150: learn: 24.2245938 total: 1m 21s remaining: 7m 37s 151: learn: 24.2034883 total: 1m 21s remaining: 7m 37s 152: learn: 24.2018801 total: 1m 22s remaining: 7m 36s 153: learn: 24.1984362 total: 1m 22s remaining: 7m 35s 154: learn: 24.1892868 total: 1m 23s remaining: 7m 34s 155: learn: 24.1811730 total: 1m 23s remaining: 7m 34s 156: learn: 24.1247160 total: 1m 24s remaining: 7m 33s 157: learn: 24.1235172 total: 1m 24s remaining: 7m 32s 158: learn: 24.1195710 total: 1m 25s remaining: 7m 31s 159: learn: 24.0938600 total: 1m 25s remaining: 7m 30s 160: learn: 24.0486681 total: 1m 26s remaining: 7m 29s 161: learn: 24.0224449 total: 1m 26s remaining: 7m 28s 162: learn: 24.0151956 total: 1m 27s remaining: 7m 27s 163: learn: 24.0151813 total: 1m 27s remaining: 7m 26s 164: learn: 24.0142314 total: 1m 28s remaining: 7m 25s 165: learn: 24.0067178 total: 1m 28s remaining: 7m 24s 166: learn: 24.0056518 total: 1m 28s remaining: 7m 23s 167: learn: 24.0014614 total: 1m 29s remaining: 7m 22s 168: learn: 23.9949149 total: 1m 29s remaining: 7m 22s 169: learn: 23.9597603 total: 1m 30s remaining: 7m 21s 170: learn: 23.9384054 total: 1m 30s remaining: 7m 20s 171: learn: 23.9209449 total: 1m 31s remaining: 7m 19s 172: learn: 23.9199428 total: 1m 31s remaining: 7m 19s 173: learn: 23.8540362 total: 1m 32s remaining: 7m 18s 174: learn: 23.8540253 total: 1m 32s remaining: 7m 17s 175: learn: 23.8371117 total: 1m 33s remaining: 7m 16s 176: learn: 23.8186173 total: 1m 33s remaining: 7m 15s 177: learn: 23.8114983 total: 1m 34s remaining: 7m 14s 178: learn: 23.7855843 total: 1m 34s remaining: 7m 14s 179: learn: 23.7737263 total: 1m 35s remaining: 7m 13s 180: learn: 23.7728121 total: 1m 35s remaining: 7m 12s 181: learn: 23.7651993 total: 1m 36s remaining: 7m 12s 182: learn: 23.7331195 total: 1m 36s remaining: 7m 11s 183: learn: 23.7306529 total: 1m 37s remaining: 7m 11s 184: learn: 23.6988789 total: 1m 37s remaining: 7m 10s 185: learn: 23.6636114 total: 1m 38s remaining: 7m 10s 186: learn: 23.6392165 total: 1m 38s remaining: 7m 9s 187: learn: 23.6201474 total: 1m 39s remaining: 7m 9s 188: learn: 23.5937527 total: 1m 39s remaining: 7m 9s 189: learn: 23.5601666 total: 1m 40s remaining: 7m 8s 190: learn: 23.5525397 total: 1m 41s remaining: 7m 8s 191: learn: 23.5352741 total: 1m 41s remaining: 7m 7s 192: learn: 23.5084743 total: 1m 42s remaining: 7m 7s 193: learn: 23.4913344 total: 1m 42s remaining: 7m 6s 194: learn: 23.4863695 total: 1m 43s remaining: 7m 5s 195: learn: 23.4828945 total: 1m 43s remaining: 7m 5s 196: learn: 23.4819075 total: 1m 44s remaining: 7m 4s 197: learn: 23.4716506 total: 1m 44s remaining: 7m 4s 198: learn: 23.4484574 total: 1m 45s remaining: 7m 3s 199: learn: 23.4204294 total: 1m 45s remaining: 7m 3s 200: learn: 23.4196303 total: 1m 46s remaining: 7m 2s 201: learn: 23.4039253 total: 1m 46s remaining: 7m 1s 202: learn: 23.4017784 total: 1m 47s remaining: 7m 1s 203: learn: 23.4003924 total: 1m 47s remaining: 7m 204: learn: 23.3739777 total: 1m 48s remaining: 6m 59s 205: learn: 23.3713461 total: 1m 48s remaining: 6m 58s 206: learn: 23.3330499 total: 1m 49s remaining: 6m 58s 207: learn: 23.3254500 total: 1m 49s remaining: 6m 57s 208: learn: 23.3185028 total: 1m 50s remaining: 6m 56s 209: learn: 23.2921978 total: 1m 50s remaining: 6m 56s 210: learn: 23.2602862 total: 1m 51s remaining: 6m 55s 211: learn: 23.2121239 total: 1m 51s remaining: 6m 54s 212: learn: 23.2099380 total: 1m 52s remaining: 6m 53s 213: learn: 23.1739513 total: 1m 52s remaining: 6m 53s 214: learn: 23.1721438 total: 1m 52s remaining: 6m 52s 215: learn: 23.1707763 total: 1m 53s remaining: 6m 51s 216: learn: 23.1499927 total: 1m 53s remaining: 6m 50s 217: learn: 23.1394898 total: 1m 54s remaining: 6m 50s 218: learn: 23.1243236 total: 1m 54s remaining: 6m 49s 219: learn: 23.1018684 total: 1m 55s remaining: 6m 48s 220: learn: 23.0900634 total: 1m 55s remaining: 6m 48s 221: learn: 23.0842643 total: 1m 56s remaining: 6m 47s 222: learn: 23.0473764 total: 1m 56s remaining: 6m 47s 223: learn: 23.0404553 total: 1m 57s remaining: 6m 46s 224: learn: 23.0332526 total: 1m 57s remaining: 6m 45s 225: learn: 23.0318993 total: 1m 58s remaining: 6m 45s 226: learn: 23.0012116 total: 1m 58s remaining: 6m 44s 227: learn: 22.9816688 total: 1m 59s remaining: 6m 43s 228: learn: 22.9496694 total: 1m 59s remaining: 6m 42s 229: learn: 22.9337281 total: 2m remaining: 6m 42s 230: learn: 22.9298736 total: 2m remaining: 6m 41s 231: learn: 22.9271131 total: 2m 1s remaining: 6m 41s 232: learn: 22.8949823 total: 2m 1s remaining: 6m 40s 233: learn: 22.8870222 total: 2m 2s remaining: 6m 39s 234: learn: 22.8842864 total: 2m 2s remaining: 6m 39s 235: learn: 22.8201388 total: 2m 3s remaining: 6m 38s 236: learn: 22.8160140 total: 2m 3s remaining: 6m 38s 237: learn: 22.7959829 total: 2m 4s remaining: 6m 37s 238: learn: 22.7665224 total: 2m 4s remaining: 6m 36s 239: learn: 22.7661570 total: 2m 5s remaining: 6m 36s 240: learn: 22.7438938 total: 2m 5s remaining: 6m 35s 241: learn: 22.7155798 total: 2m 6s remaining: 6m 35s 242: learn: 22.7124790 total: 2m 6s remaining: 6m 34s 243: learn: 22.6751986 total: 2m 7s remaining: 6m 34s 244: learn: 22.6690142 total: 2m 7s remaining: 6m 33s 245: learn: 22.6536795 total: 2m 8s remaining: 6m 33s 246: learn: 22.6351651 total: 2m 8s remaining: 6m 32s 247: learn: 22.6124454 total: 2m 9s remaining: 6m 31s 248: learn: 22.6061485 total: 2m 9s remaining: 6m 31s 249: learn: 22.6051605 total: 2m 10s remaining: 6m 30s 250: learn: 22.5778938 total: 2m 10s remaining: 6m 29s 251: learn: 22.5705864 total: 2m 11s remaining: 6m 29s 252: learn: 22.5604160 total: 2m 11s remaining: 6m 28s 253: learn: 22.5494668 total: 2m 12s remaining: 6m 27s 254: learn: 22.5362326 total: 2m 12s remaining: 6m 27s 255: learn: 22.5012974 total: 2m 12s remaining: 6m 26s 256: learn: 22.4732771 total: 2m 13s remaining: 6m 25s 257: learn: 22.4719553 total: 2m 13s remaining: 6m 25s 258: learn: 22.4441354 total: 2m 14s remaining: 6m 24s 259: learn: 22.4304229 total: 2m 14s remaining: 6m 24s 260: learn: 22.4107113 total: 2m 15s remaining: 6m 23s 261: learn: 22.3897651 total: 2m 15s remaining: 6m 23s 262: learn: 22.3777344 total: 2m 16s remaining: 6m 22s 263: learn: 22.3592352 total: 2m 16s remaining: 6m 21s 264: learn: 22.3493113 total: 2m 17s remaining: 6m 21s 265: learn: 22.3364456 total: 2m 17s remaining: 6m 20s 266: learn: 22.3354836 total: 2m 18s remaining: 6m 19s 267: learn: 22.3345998 total: 2m 18s remaining: 6m 19s 268: learn: 22.3146473 total: 2m 19s remaining: 6m 18s 269: learn: 22.3003275 total: 2m 19s remaining: 6m 17s 270: learn: 22.2846458 total: 2m 20s remaining: 6m 17s 271: learn: 22.2791975 total: 2m 20s remaining: 6m 16s 272: learn: 22.2706949 total: 2m 21s remaining: 6m 15s 273: learn: 22.2684467 total: 2m 21s remaining: 6m 15s 274: learn: 22.2611253 total: 2m 22s remaining: 6m 14s 275: learn: 22.2538102 total: 2m 22s remaining: 6m 13s 276: learn: 22.2433106 total: 2m 22s remaining: 6m 13s 277: learn: 22.2221577 total: 2m 23s remaining: 6m 12s 278: learn: 22.2195008 total: 2m 23s remaining: 6m 11s 279: learn: 22.1959802 total: 2m 24s remaining: 6m 11s 280: learn: 22.1952267 total: 2m 24s remaining: 6m 10s 281: learn: 22.1935153 total: 2m 25s remaining: 6m 10s 282: learn: 22.1748022 total: 2m 25s remaining: 6m 9s 283: learn: 22.1639233 total: 2m 26s remaining: 6m 8s 284: learn: 22.1251726 total: 2m 26s remaining: 6m 8s 285: learn: 22.1138439 total: 2m 27s remaining: 6m 7s 286: learn: 22.0984138 total: 2m 27s remaining: 6m 7s 287: learn: 22.0922711 total: 2m 28s remaining: 6m 6s 288: learn: 22.0755391 total: 2m 28s remaining: 6m 5s 289: learn: 22.0747357 total: 2m 29s remaining: 6m 5s 290: learn: 22.0309697 total: 2m 29s remaining: 6m 4s 291: learn: 22.0159251 total: 2m 30s remaining: 6m 4s 292: learn: 22.0148116 total: 2m 30s remaining: 6m 3s 293: learn: 21.9843638 total: 2m 31s remaining: 6m 3s 294: learn: 21.9747351 total: 2m 31s remaining: 6m 2s 295: learn: 21.9740728 total: 2m 32s remaining: 6m 2s 296: learn: 21.9693104 total: 2m 32s remaining: 6m 1s 297: learn: 21.9684771 total: 2m 33s remaining: 6m 1s 298: learn: 21.9577827 total: 2m 33s remaining: 6m 299: learn: 21.9401313 total: 2m 34s remaining: 5m 59s 300: learn: 21.9331703 total: 2m 34s remaining: 5m 59s 301: learn: 21.9251689 total: 2m 35s remaining: 5m 58s 302: learn: 21.9227709 total: 2m 35s remaining: 5m 58s 303: learn: 21.9215984 total: 2m 36s remaining: 5m 57s 304: learn: 21.8686191 total: 2m 36s remaining: 5m 57s 305: learn: 21.8642068 total: 2m 37s remaining: 5m 56s 306: learn: 21.8636794 total: 2m 37s remaining: 5m 55s 307: learn: 21.8513036 total: 2m 38s remaining: 5m 55s 308: learn: 21.8507605 total: 2m 38s remaining: 5m 54s 309: learn: 21.8495373 total: 2m 39s remaining: 5m 54s 310: learn: 21.8396246 total: 2m 39s remaining: 5m 53s 311: learn: 21.8390678 total: 2m 40s remaining: 5m 52s 312: learn: 21.8355183 total: 2m 40s remaining: 5m 52s 313: learn: 21.8350989 total: 2m 41s remaining: 5m 51s 314: learn: 21.8341724 total: 2m 41s remaining: 5m 51s 315: learn: 21.8256561 total: 2m 42s remaining: 5m 50s 316: learn: 21.8166065 total: 2m 42s remaining: 5m 50s 317: learn: 21.8085208 total: 2m 42s remaining: 5m 49s 318: learn: 21.8082432 total: 2m 43s remaining: 5m 49s 319: learn: 21.7992213 total: 2m 43s remaining: 5m 48s 320: learn: 21.7886326 total: 2m 44s remaining: 5m 47s 321: learn: 21.7833303 total: 2m 44s remaining: 5m 47s 322: learn: 21.7715199 total: 2m 45s remaining: 5m 46s 323: learn: 21.7702489 total: 2m 45s remaining: 5m 45s 324: learn: 21.7454369 total: 2m 46s remaining: 5m 45s 325: learn: 21.7360362 total: 2m 46s remaining: 5m 44s 326: learn: 21.7189017 total: 2m 47s remaining: 5m 44s 327: learn: 21.7067280 total: 2m 47s remaining: 5m 43s 328: learn: 21.7034029 total: 2m 48s remaining: 5m 42s 329: learn: 21.6994343 total: 2m 48s remaining: 5m 42s 330: learn: 21.6755742 total: 2m 49s remaining: 5m 41s 331: learn: 21.6719804 total: 2m 49s remaining: 5m 41s 332: learn: 21.6486022 total: 2m 49s remaining: 5m 40s 333: learn: 21.6360067 total: 2m 50s remaining: 5m 39s 334: learn: 21.5994658 total: 2m 50s remaining: 5m 39s 335: learn: 21.5969715 total: 2m 51s remaining: 5m 38s 336: learn: 21.5816721 total: 2m 51s remaining: 5m 38s 337: learn: 21.5784734 total: 2m 52s remaining: 5m 37s 338: learn: 21.5754327 total: 2m 52s remaining: 5m 36s 339: learn: 21.5349771 total: 2m 53s remaining: 5m 36s 340: learn: 21.5275495 total: 2m 53s remaining: 5m 35s 341: learn: 21.4939319 total: 2m 54s remaining: 5m 35s 342: learn: 21.4707465 total: 2m 54s remaining: 5m 34s 343: learn: 21.4691812 total: 2m 55s remaining: 5m 33s 344: learn: 21.4485513 total: 2m 55s remaining: 5m 33s 345: learn: 21.4069451 total: 2m 56s remaining: 5m 32s 346: learn: 21.3751849 total: 2m 56s remaining: 5m 32s 347: learn: 21.3727581 total: 2m 57s remaining: 5m 31s 348: learn: 21.3699395 total: 2m 57s remaining: 5m 31s 349: learn: 21.3635000 total: 2m 58s remaining: 5m 30s 350: learn: 21.3516630 total: 2m 58s remaining: 5m 30s 351: learn: 21.3449694 total: 2m 59s remaining: 5m 29s 352: learn: 21.3315123 total: 2m 59s remaining: 5m 29s 353: learn: 21.3306351 total: 3m remaining: 5m 28s 354: learn: 21.3221705 total: 3m remaining: 5m 27s 355: learn: 21.3217945 total: 3m remaining: 5m 27s 356: learn: 21.3083326 total: 3m 1s remaining: 5m 26s 357: learn: 21.2954312 total: 3m 1s remaining: 5m 26s 358: learn: 21.2927241 total: 3m 2s remaining: 5m 25s 359: learn: 21.2700492 total: 3m 3s remaining: 5m 25s 360: learn: 21.2563104 total: 3m 3s remaining: 5m 24s 361: learn: 21.2446470 total: 3m 3s remaining: 5m 24s 362: learn: 21.2392902 total: 3m 4s remaining: 5m 23s 363: learn: 21.2313414 total: 3m 4s remaining: 5m 23s 364: learn: 21.2251570 total: 3m 5s remaining: 5m 22s 365: learn: 21.2207856 total: 3m 5s remaining: 5m 22s 366: learn: 21.2194263 total: 3m 6s remaining: 5m 21s 367: learn: 21.2153790 total: 3m 6s remaining: 5m 21s 368: learn: 21.1868223 total: 3m 7s remaining: 5m 20s 369: learn: 21.1821525 total: 3m 8s remaining: 5m 20s 370: learn: 21.1513688 total: 3m 8s remaining: 5m 19s 371: learn: 21.1471882 total: 3m 9s remaining: 5m 19s 372: learn: 21.1420183 total: 3m 9s remaining: 5m 18s 373: learn: 21.1381216 total: 3m 10s remaining: 5m 18s 374: learn: 21.1298514 total: 3m 10s remaining: 5m 17s 375: learn: 21.1167861 total: 3m 11s remaining: 5m 17s 376: learn: 21.1152454 total: 3m 11s remaining: 5m 16s 377: learn: 21.0901762 total: 3m 12s remaining: 5m 16s 378: learn: 21.0833515 total: 3m 12s remaining: 5m 15s 379: learn: 21.0746731 total: 3m 13s remaining: 5m 15s 380: learn: 21.0501280 total: 3m 13s remaining: 5m 14s 381: learn: 21.0394883 total: 3m 14s remaining: 5m 14s 382: learn: 21.0231015 total: 3m 14s remaining: 5m 13s 383: learn: 21.0002056 total: 3m 15s remaining: 5m 13s 384: learn: 20.9726656 total: 3m 15s remaining: 5m 12s 385: learn: 20.9668932 total: 3m 16s remaining: 5m 12s 386: learn: 20.9473384 total: 3m 16s remaining: 5m 11s 387: learn: 20.9453801 total: 3m 17s remaining: 5m 11s 388: learn: 20.9349392 total: 3m 17s remaining: 5m 10s 389: learn: 20.9158672 total: 3m 18s remaining: 5m 9s 390: learn: 20.9136881 total: 3m 18s remaining: 5m 9s 391: learn: 20.8986318 total: 3m 19s remaining: 5m 8s 392: learn: 20.8916862 total: 3m 19s remaining: 5m 8s 393: learn: 20.8857503 total: 3m 20s remaining: 5m 7s 394: learn: 20.8789880 total: 3m 20s remaining: 5m 7s 395: learn: 20.8787042 total: 3m 21s remaining: 5m 6s 396: learn: 20.8735998 total: 3m 21s remaining: 5m 6s 397: learn: 20.8630634 total: 3m 21s remaining: 5m 5s 398: learn: 20.8627025 total: 3m 22s remaining: 5m 4s 399: learn: 20.8600571 total: 3m 22s remaining: 5m 4s 400: learn: 20.8556865 total: 3m 23s remaining: 5m 3s 401: learn: 20.8393392 total: 3m 23s remaining: 5m 3s 402: learn: 20.8349776 total: 3m 24s remaining: 5m 2s 403: learn: 20.8344969 total: 3m 24s remaining: 5m 2s 404: learn: 20.8198682 total: 3m 25s remaining: 5m 1s 405: learn: 20.8157194 total: 3m 25s remaining: 5m 1s 406: learn: 20.8056923 total: 3m 26s remaining: 5m 407: learn: 20.7948153 total: 3m 26s remaining: 5m 408: learn: 20.7784471 total: 3m 27s remaining: 4m 59s 409: learn: 20.7781645 total: 3m 28s remaining: 4m 59s 410: learn: 20.7740512 total: 3m 28s remaining: 4m 58s 411: learn: 20.7686284 total: 3m 29s remaining: 4m 58s 412: learn: 20.7657193 total: 3m 29s remaining: 4m 57s 413: learn: 20.7594768 total: 3m 30s remaining: 4m 57s 414: learn: 20.7549351 total: 3m 30s remaining: 4m 56s 415: learn: 20.7416547 total: 3m 31s remaining: 4m 56s 416: learn: 20.7398790 total: 3m 31s remaining: 4m 55s 417: learn: 20.7379730 total: 3m 32s remaining: 4m 55s 418: learn: 20.7358712 total: 3m 32s remaining: 4m 54s 419: learn: 20.7223127 total: 3m 33s remaining: 4m 54s 420: learn: 20.7200128 total: 3m 33s remaining: 4m 53s 421: learn: 20.7084442 total: 3m 34s remaining: 4m 53s 422: learn: 20.6999239 total: 3m 34s remaining: 4m 52s 423: learn: 20.6955845 total: 3m 35s remaining: 4m 52s 424: learn: 20.6706508 total: 3m 35s remaining: 4m 51s 425: learn: 20.6661664 total: 3m 36s remaining: 4m 51s 426: learn: 20.6610538 total: 3m 36s remaining: 4m 50s 427: learn: 20.6412022 total: 3m 37s remaining: 4m 50s 428: learn: 20.6165420 total: 3m 37s remaining: 4m 49s 429: learn: 20.6150506 total: 3m 38s remaining: 4m 49s 430: learn: 20.6043328 total: 3m 38s remaining: 4m 48s 431: learn: 20.6041659 total: 3m 39s remaining: 4m 48s 432: learn: 20.6027699 total: 3m 39s remaining: 4m 47s 433: learn: 20.5928448 total: 3m 40s remaining: 4m 47s 434: learn: 20.5897734 total: 3m 40s remaining: 4m 46s 435: learn: 20.5886528 total: 3m 41s remaining: 4m 46s 436: learn: 20.5851005 total: 3m 41s remaining: 4m 45s 437: learn: 20.5765099 total: 3m 42s remaining: 4m 44s 438: learn: 20.5633996 total: 3m 42s remaining: 4m 44s 439: learn: 20.5530082 total: 3m 42s remaining: 4m 43s 440: learn: 20.5505828 total: 3m 43s remaining: 4m 43s 441: learn: 20.5434356 total: 3m 43s remaining: 4m 42s 442: learn: 20.5307158 total: 3m 44s remaining: 4m 42s 443: learn: 20.5124713 total: 3m 44s remaining: 4m 41s 444: learn: 20.5114730 total: 3m 45s remaining: 4m 41s 445: learn: 20.5098965 total: 3m 45s remaining: 4m 40s 446: learn: 20.4817947 total: 3m 46s remaining: 4m 39s 447: learn: 20.4792062 total: 3m 46s remaining: 4m 39s 448: learn: 20.4703805 total: 3m 47s remaining: 4m 38s 449: learn: 20.4692307 total: 3m 47s remaining: 4m 38s 450: learn: 20.4672475 total: 3m 48s remaining: 4m 37s 451: learn: 20.4604873 total: 3m 48s remaining: 4m 37s 452: learn: 20.4584665 total: 3m 49s remaining: 4m 36s 453: learn: 20.4432936 total: 3m 49s remaining: 4m 36s 454: learn: 20.4329620 total: 3m 50s remaining: 4m 35s 455: learn: 20.4327382 total: 3m 50s remaining: 4m 35s 456: learn: 20.4220912 total: 3m 51s remaining: 4m 34s 457: learn: 20.3825262 total: 3m 51s remaining: 4m 34s 458: learn: 20.3749619 total: 3m 52s remaining: 4m 33s 459: learn: 20.3655721 total: 3m 52s remaining: 4m 33s 460: learn: 20.3515227 total: 3m 53s remaining: 4m 32s 461: learn: 20.3421076 total: 3m 53s remaining: 4m 32s 462: learn: 20.3352728 total: 3m 54s remaining: 4m 31s 463: learn: 20.3349851 total: 3m 54s remaining: 4m 31s 464: learn: 20.3313364 total: 3m 55s remaining: 4m 30s 465: learn: 20.3235754 total: 3m 55s remaining: 4m 30s 466: learn: 20.3197533 total: 3m 56s remaining: 4m 29s 467: learn: 20.3145297 total: 3m 56s remaining: 4m 29s 468: learn: 20.3064542 total: 3m 57s remaining: 4m 28s 469: learn: 20.2927566 total: 3m 57s remaining: 4m 28s 470: learn: 20.2823941 total: 3m 58s remaining: 4m 27s 471: learn: 20.2671297 total: 3m 58s remaining: 4m 27s 472: learn: 20.2666559 total: 3m 59s remaining: 4m 26s 473: learn: 20.2592208 total: 3m 59s remaining: 4m 26s 474: learn: 20.2549117 total: 4m remaining: 4m 25s 475: learn: 20.2482705 total: 4m remaining: 4m 25s 476: learn: 20.2424393 total: 4m 1s remaining: 4m 24s 477: learn: 20.2336900 total: 4m 1s remaining: 4m 23s 478: learn: 20.2312606 total: 4m 2s remaining: 4m 23s 479: learn: 20.2209729 total: 4m 2s remaining: 4m 22s 480: learn: 20.2186885 total: 4m 3s remaining: 4m 22s 481: learn: 20.2097882 total: 4m 3s remaining: 4m 21s 482: learn: 20.2071597 total: 4m 4s remaining: 4m 21s 483: learn: 20.2010148 total: 4m 4s remaining: 4m 20s 484: learn: 20.1957757 total: 4m 5s remaining: 4m 20s 485: learn: 20.1847178 total: 4m 5s remaining: 4m 19s 486: learn: 20.1834504 total: 4m 6s remaining: 4m 19s 487: learn: 20.1831917 total: 4m 6s remaining: 4m 18s 488: learn: 20.1808519 total: 4m 7s remaining: 4m 18s 489: learn: 20.1795119 total: 4m 7s remaining: 4m 17s 490: learn: 20.1783886 total: 4m 8s remaining: 4m 17s 491: learn: 20.1751114 total: 4m 8s remaining: 4m 17s 492: learn: 20.1735240 total: 4m 9s remaining: 4m 16s 493: learn: 20.1560369 total: 4m 10s remaining: 4m 16s 494: learn: 20.1514851 total: 4m 10s remaining: 4m 15s 495: learn: 20.1481900 total: 4m 11s remaining: 4m 15s 496: learn: 20.1362265 total: 4m 12s remaining: 4m 15s 497: learn: 20.1297406 total: 4m 12s remaining: 4m 14s 498: learn: 20.1269037 total: 4m 13s remaining: 4m 14s 499: learn: 20.1188524 total: 4m 14s remaining: 4m 14s 500: learn: 20.1116312 total: 4m 14s remaining: 4m 13s 501: learn: 20.1050980 total: 4m 15s remaining: 4m 13s 502: learn: 20.0994641 total: 4m 15s remaining: 4m 12s 503: learn: 20.0912455 total: 4m 16s remaining: 4m 12s 504: learn: 20.0835050 total: 4m 16s remaining: 4m 11s 505: learn: 20.0786633 total: 4m 17s remaining: 4m 11s 506: learn: 20.0735476 total: 4m 17s remaining: 4m 10s 507: learn: 20.0732660 total: 4m 18s remaining: 4m 10s 508: learn: 20.0712958 total: 4m 18s remaining: 4m 9s 509: learn: 20.0552487 total: 4m 19s remaining: 4m 9s 510: learn: 20.0523782 total: 4m 20s remaining: 4m 8s 511: learn: 20.0263161 total: 4m 20s remaining: 4m 8s 512: learn: 20.0110237 total: 4m 20s remaining: 4m 7s 513: learn: 20.0078860 total: 4m 21s remaining: 4m 7s 514: learn: 19.9948214 total: 4m 22s remaining: 4m 6s 515: learn: 19.9935142 total: 4m 22s remaining: 4m 6s 516: learn: 19.9879410 total: 4m 23s remaining: 4m 5s 517: learn: 19.9839122 total: 4m 23s remaining: 4m 5s 518: learn: 19.9786361 total: 4m 24s remaining: 4m 4s 519: learn: 19.9771291 total: 4m 24s remaining: 4m 4s 520: learn: 19.9738936 total: 4m 25s remaining: 4m 3s 521: learn: 19.9686730 total: 4m 25s remaining: 4m 3s 522: learn: 19.9671774 total: 4m 26s remaining: 4m 3s 523: learn: 19.9669608 total: 4m 26s remaining: 4m 2s 524: learn: 19.9600126 total: 4m 27s remaining: 4m 2s 525: learn: 19.9529865 total: 4m 28s remaining: 4m 1s 526: learn: 19.9470893 total: 4m 28s remaining: 4m 1s 527: learn: 19.9459600 total: 4m 29s remaining: 4m 528: learn: 19.9448260 total: 4m 29s remaining: 4m 529: learn: 19.9154392 total: 4m 30s remaining: 4m 530: learn: 19.9141056 total: 4m 31s remaining: 3m 59s 531: learn: 19.8954514 total: 4m 31s remaining: 3m 59s 532: learn: 19.8941742 total: 4m 32s remaining: 3m 58s 533: learn: 19.8932634 total: 4m 33s remaining: 3m 58s 534: learn: 19.8842260 total: 4m 33s remaining: 3m 57s 535: learn: 19.8818971 total: 4m 34s remaining: 3m 57s 536: learn: 19.8711693 total: 4m 35s remaining: 3m 57s 537: learn: 19.8694478 total: 4m 35s remaining: 3m 56s 538: learn: 19.8692218 total: 4m 36s remaining: 3m 56s 539: learn: 19.8682897 total: 4m 36s remaining: 3m 55s 540: learn: 19.8535847 total: 4m 38s remaining: 3m 56s 541: learn: 19.8322858 total: 4m 39s remaining: 3m 55s 542: learn: 19.8116582 total: 4m 39s remaining: 3m 55s 543: learn: 19.8114341 total: 4m 40s remaining: 3m 54s 544: learn: 19.8097004 total: 4m 40s remaining: 3m 54s 545: learn: 19.8084650 total: 4m 41s remaining: 3m 53s 546: learn: 19.7994699 total: 4m 41s remaining: 3m 53s 547: learn: 19.7948197 total: 4m 42s remaining: 3m 52s 548: learn: 19.7903527 total: 4m 42s remaining: 3m 52s 549: learn: 19.7890579 total: 4m 43s remaining: 3m 51s 550: learn: 19.7858372 total: 4m 43s remaining: 3m 51s 551: learn: 19.7683759 total: 4m 44s remaining: 3m 50s 552: learn: 19.7601560 total: 4m 44s remaining: 3m 50s 553: learn: 19.7502458 total: 4m 45s remaining: 3m 49s 554: learn: 19.7428566 total: 4m 45s remaining: 3m 49s 555: learn: 19.7318092 total: 4m 46s remaining: 3m 48s 556: learn: 19.7308873 total: 4m 47s remaining: 3m 48s 557: learn: 19.7304709 total: 4m 47s remaining: 3m 47s 558: learn: 19.7280574 total: 4m 48s remaining: 3m 47s 559: learn: 19.7229622 total: 4m 48s remaining: 3m 46s 560: learn: 19.7090523 total: 4m 49s remaining: 3m 46s 561: learn: 19.7003441 total: 4m 49s remaining: 3m 45s 562: learn: 19.6955655 total: 4m 50s remaining: 3m 45s 563: learn: 19.6873815 total: 4m 51s remaining: 3m 45s 564: learn: 19.6832542 total: 4m 51s remaining: 3m 44s 565: learn: 19.6761568 total: 4m 52s remaining: 3m 44s 566: learn: 19.6721899 total: 4m 53s remaining: 3m 43s 567: learn: 19.6547705 total: 4m 53s remaining: 3m 43s 568: learn: 19.6521255 total: 4m 53s remaining: 3m 42s 569: learn: 19.6479232 total: 4m 54s remaining: 3m 42s 570: learn: 19.6360260 total: 4m 54s remaining: 3m 41s 571: learn: 19.6333644 total: 4m 55s remaining: 3m 41s 572: learn: 19.6300992 total: 4m 55s remaining: 3m 40s 573: learn: 19.6260833 total: 4m 56s remaining: 3m 39s 574: learn: 19.6181622 total: 4m 56s remaining: 3m 39s 575: learn: 19.6041546 total: 4m 57s remaining: 3m 39s 576: learn: 19.6003028 total: 4m 58s remaining: 3m 38s 577: learn: 19.5930999 total: 4m 58s remaining: 3m 37s 578: learn: 19.5698709 total: 4m 58s remaining: 3m 37s 579: learn: 19.5654646 total: 4m 59s remaining: 3m 36s 580: learn: 19.5643079 total: 4m 59s remaining: 3m 36s 581: learn: 19.5633297 total: 5m remaining: 3m 35s 582: learn: 19.5532670 total: 5m remaining: 3m 35s 583: learn: 19.5516073 total: 5m 1s remaining: 3m 34s 584: learn: 19.5455754 total: 5m 1s remaining: 3m 34s 585: learn: 19.5400242 total: 5m 2s remaining: 3m 33s 586: learn: 19.5356955 total: 5m 2s remaining: 3m 33s 587: learn: 19.5323058 total: 5m 3s remaining: 3m 32s 588: learn: 19.5321307 total: 5m 3s remaining: 3m 32s 589: learn: 19.5293981 total: 5m 4s remaining: 3m 31s 590: learn: 19.5260027 total: 5m 4s remaining: 3m 30s 591: learn: 19.5243109 total: 5m 5s remaining: 3m 30s 592: learn: 19.5235026 total: 5m 5s remaining: 3m 29s 593: learn: 19.5209184 total: 5m 6s remaining: 3m 29s 594: learn: 19.5169323 total: 5m 6s remaining: 3m 28s 595: learn: 19.5122659 total: 5m 7s remaining: 3m 28s 596: learn: 19.5116824 total: 5m 8s remaining: 3m 27s 597: learn: 19.5110745 total: 5m 8s remaining: 3m 27s 598: learn: 19.5108041 total: 5m 9s remaining: 3m 27s 599: learn: 19.5076477 total: 5m 9s remaining: 3m 26s 600: learn: 19.5057941 total: 5m 10s remaining: 3m 26s 601: learn: 19.5040549 total: 5m 11s remaining: 3m 25s 602: learn: 19.5035947 total: 5m 11s remaining: 3m 25s 603: learn: 19.5023873 total: 5m 12s remaining: 3m 24s 604: learn: 19.4965467 total: 5m 12s remaining: 3m 24s 605: learn: 19.4915164 total: 5m 13s remaining: 3m 23s 606: learn: 19.4903841 total: 5m 13s remaining: 3m 22s 607: learn: 19.4902656 total: 5m 13s remaining: 3m 22s 608: learn: 19.4896533 total: 5m 14s remaining: 3m 21s 609: learn: 19.4812312 total: 5m 14s remaining: 3m 21s 610: learn: 19.4798178 total: 5m 15s remaining: 3m 20s 611: learn: 19.4764997 total: 5m 15s remaining: 3m 20s 612: learn: 19.4658315 total: 5m 16s remaining: 3m 19s 613: learn: 19.4639540 total: 5m 16s remaining: 3m 19s 614: learn: 19.4573846 total: 5m 17s remaining: 3m 18s 615: learn: 19.4546516 total: 5m 17s remaining: 3m 18s 616: learn: 19.4533132 total: 5m 18s remaining: 3m 17s 617: learn: 19.4488539 total: 5m 18s remaining: 3m 17s 618: learn: 19.4476815 total: 5m 19s remaining: 3m 16s 619: learn: 19.4347174 total: 5m 19s remaining: 3m 15s 620: learn: 19.4312588 total: 5m 20s remaining: 3m 15s 621: learn: 19.4275725 total: 5m 20s remaining: 3m 14s 622: learn: 19.4254384 total: 5m 21s remaining: 3m 14s 623: learn: 19.4167555 total: 5m 21s remaining: 3m 13s 624: learn: 19.4139444 total: 5m 22s remaining: 3m 13s 625: learn: 19.4110111 total: 5m 22s remaining: 3m 12s 626: learn: 19.4077412 total: 5m 23s remaining: 3m 12s 627: learn: 19.4063661 total: 5m 23s remaining: 3m 11s 628: learn: 19.4024885 total: 5m 24s remaining: 3m 11s 629: learn: 19.4013462 total: 5m 24s remaining: 3m 10s 630: learn: 19.3974357 total: 5m 25s remaining: 3m 10s 631: learn: 19.3840092 total: 5m 25s remaining: 3m 9s 632: learn: 19.3828905 total: 5m 26s remaining: 3m 9s 633: learn: 19.3684626 total: 5m 26s remaining: 3m 8s 634: learn: 19.3661021 total: 5m 27s remaining: 3m 8s 635: learn: 19.3636159 total: 5m 27s remaining: 3m 7s 636: learn: 19.3619127 total: 5m 28s remaining: 3m 7s 637: learn: 19.3538291 total: 5m 28s remaining: 3m 6s 638: learn: 19.3498628 total: 5m 29s remaining: 3m 6s 639: learn: 19.3429461 total: 5m 30s remaining: 3m 5s 640: learn: 19.3307032 total: 5m 30s remaining: 3m 5s 641: learn: 19.3216536 total: 5m 31s remaining: 3m 4s 642: learn: 19.3094666 total: 5m 31s remaining: 3m 4s 643: learn: 19.3055415 total: 5m 32s remaining: 3m 3s 644: learn: 19.2979040 total: 5m 32s remaining: 3m 3s 645: learn: 19.2845582 total: 5m 33s remaining: 3m 2s 646: learn: 19.2838962 total: 5m 33s remaining: 3m 2s 647: learn: 19.2811758 total: 5m 34s remaining: 3m 1s 648: learn: 19.2789470 total: 5m 34s remaining: 3m 1s 649: learn: 19.2752884 total: 5m 35s remaining: 3m 650: learn: 19.2710020 total: 5m 35s remaining: 2m 59s 651: learn: 19.2638666 total: 5m 36s remaining: 2m 59s 652: learn: 19.2537965 total: 5m 36s remaining: 2m 58s 653: learn: 19.2491229 total: 5m 37s remaining: 2m 58s 654: learn: 19.2465593 total: 5m 37s remaining: 2m 57s 655: learn: 19.2461476 total: 5m 38s remaining: 2m 57s 656: learn: 19.2436105 total: 5m 38s remaining: 2m 56s 657: learn: 19.2422607 total: 5m 39s remaining: 2m 56s 658: learn: 19.2421626 total: 5m 39s remaining: 2m 55s 659: learn: 19.2346367 total: 5m 40s remaining: 2m 55s 660: learn: 19.2279978 total: 5m 40s remaining: 2m 54s 661: learn: 19.2231730 total: 5m 41s remaining: 2m 54s 662: learn: 19.2217121 total: 5m 41s remaining: 2m 53s 663: learn: 19.2184569 total: 5m 42s remaining: 2m 53s 664: learn: 19.2173272 total: 5m 42s remaining: 2m 52s 665: learn: 19.2096337 total: 5m 42s remaining: 2m 51s 666: learn: 19.2079547 total: 5m 43s remaining: 2m 51s 667: learn: 19.1974128 total: 5m 43s remaining: 2m 50s 668: learn: 19.1847534 total: 5m 44s remaining: 2m 50s 669: learn: 19.1817423 total: 5m 44s remaining: 2m 49s 670: learn: 19.1805056 total: 5m 45s remaining: 2m 49s 671: learn: 19.1785515 total: 5m 45s remaining: 2m 48s 672: learn: 19.1755579 total: 5m 46s remaining: 2m 48s 673: learn: 19.1666202 total: 5m 46s remaining: 2m 47s 674: learn: 19.1664590 total: 5m 47s remaining: 2m 47s 675: learn: 19.1643466 total: 5m 47s remaining: 2m 46s 676: learn: 19.1566124 total: 5m 48s remaining: 2m 46s 677: learn: 19.1503596 total: 5m 48s remaining: 2m 45s 678: learn: 19.1336689 total: 5m 49s remaining: 2m 45s 679: learn: 19.1288716 total: 5m 49s remaining: 2m 44s 680: learn: 19.1205011 total: 5m 50s remaining: 2m 44s 681: learn: 19.1180519 total: 5m 51s remaining: 2m 43s 682: learn: 19.1179483 total: 5m 51s remaining: 2m 43s 683: learn: 19.1178667 total: 5m 52s remaining: 2m 42s 684: learn: 19.1177857 total: 5m 52s remaining: 2m 42s 685: learn: 19.1150547 total: 5m 53s remaining: 2m 41s 686: learn: 19.1105930 total: 5m 53s remaining: 2m 41s 687: learn: 19.1099839 total: 5m 54s remaining: 2m 40s 688: learn: 19.1006864 total: 5m 54s remaining: 2m 40s 689: learn: 19.0946892 total: 5m 55s remaining: 2m 39s 690: learn: 19.0934420 total: 5m 56s remaining: 2m 39s 691: learn: 19.0775362 total: 5m 56s remaining: 2m 38s 692: learn: 19.0719594 total: 5m 57s remaining: 2m 38s 693: learn: 19.0703525 total: 5m 57s remaining: 2m 37s 694: learn: 19.0616728 total: 5m 58s remaining: 2m 37s 695: learn: 19.0611182 total: 5m 59s remaining: 2m 36s 696: learn: 19.0482257 total: 5m 59s remaining: 2m 36s 697: learn: 19.0477836 total: 6m remaining: 2m 35s 698: learn: 19.0435275 total: 6m 1s remaining: 2m 35s 699: learn: 19.0353348 total: 6m 1s remaining: 2m 35s 700: learn: 19.0350282 total: 6m 2s remaining: 2m 34s 701: learn: 19.0345169 total: 6m 3s remaining: 2m 34s 702: learn: 19.0161021 total: 6m 3s remaining: 2m 33s 703: learn: 19.0038880 total: 6m 4s remaining: 2m 33s 704: learn: 19.0023627 total: 6m 4s remaining: 2m 32s 705: learn: 18.9927490 total: 6m 5s remaining: 2m 32s 706: learn: 18.9910644 total: 6m 5s remaining: 2m 31s 707: learn: 18.9895535 total: 6m 6s remaining: 2m 31s 708: learn: 18.9889751 total: 6m 7s remaining: 2m 30s 709: learn: 18.9851595 total: 6m 7s remaining: 2m 30s 710: learn: 18.9836238 total: 6m 8s remaining: 2m 29s 711: learn: 18.9817792 total: 6m 8s remaining: 2m 29s 712: learn: 18.9812048 total: 6m 8s remaining: 2m 28s 713: learn: 18.9799237 total: 6m 9s remaining: 2m 27s 714: learn: 18.9682310 total: 6m 10s remaining: 2m 27s 715: learn: 18.9511031 total: 6m 10s remaining: 2m 26s 716: learn: 18.9457869 total: 6m 11s remaining: 2m 26s 717: learn: 18.9448211 total: 6m 11s remaining: 2m 25s 718: learn: 18.9444873 total: 6m 12s remaining: 2m 25s 719: learn: 18.9407601 total: 6m 12s remaining: 2m 24s 720: learn: 18.9346381 total: 6m 13s remaining: 2m 24s 721: learn: 18.9325197 total: 6m 13s remaining: 2m 23s 722: learn: 18.9199185 total: 6m 14s remaining: 2m 23s 723: learn: 18.9182826 total: 6m 14s remaining: 2m 22s 724: learn: 18.9179912 total: 6m 15s remaining: 2m 22s 725: learn: 18.9119750 total: 6m 15s remaining: 2m 21s 726: learn: 18.9113567 total: 6m 16s remaining: 2m 21s 727: learn: 18.9102171 total: 6m 16s remaining: 2m 20s 728: learn: 18.9033090 total: 6m 17s remaining: 2m 20s 729: learn: 18.8918010 total: 6m 17s remaining: 2m 19s 730: learn: 18.8881627 total: 6m 18s remaining: 2m 19s 731: learn: 18.8867072 total: 6m 18s remaining: 2m 18s 732: learn: 18.8860471 total: 6m 19s remaining: 2m 18s 733: learn: 18.8770739 total: 6m 20s remaining: 2m 17s 734: learn: 18.8706841 total: 6m 20s remaining: 2m 17s 735: learn: 18.8679924 total: 6m 21s remaining: 2m 16s 736: learn: 18.8627925 total: 6m 21s remaining: 2m 16s 737: learn: 18.8623275 total: 6m 22s remaining: 2m 15s 738: learn: 18.8476055 total: 6m 22s remaining: 2m 15s 739: learn: 18.8455066 total: 6m 23s remaining: 2m 14s 740: learn: 18.8419293 total: 6m 23s remaining: 2m 14s 741: learn: 18.8394698 total: 6m 24s remaining: 2m 13s 742: learn: 18.8312074 total: 6m 24s remaining: 2m 13s 743: learn: 18.8308045 total: 6m 25s remaining: 2m 12s 744: learn: 18.8238947 total: 6m 25s remaining: 2m 12s 745: learn: 18.8222617 total: 6m 26s remaining: 2m 11s 746: learn: 18.8219355 total: 6m 26s remaining: 2m 11s 747: learn: 18.8177969 total: 6m 27s remaining: 2m 10s 748: learn: 18.8177176 total: 6m 27s remaining: 2m 10s 749: learn: 18.8161031 total: 6m 28s remaining: 2m 9s 750: learn: 18.8085198 total: 6m 28s remaining: 2m 8s 751: learn: 18.8045932 total: 6m 29s remaining: 2m 8s 752: learn: 18.8025465 total: 6m 30s remaining: 2m 7s 753: learn: 18.8022637 total: 6m 30s remaining: 2m 7s 754: learn: 18.7995006 total: 6m 31s remaining: 2m 7s 755: learn: 18.7990693 total: 6m 32s remaining: 2m 6s 756: learn: 18.7985687 total: 6m 32s remaining: 2m 6s 757: learn: 18.7938433 total: 6m 33s remaining: 2m 5s 758: learn: 18.7891598 total: 6m 33s remaining: 2m 5s 759: learn: 18.7857178 total: 6m 34s remaining: 2m 4s 760: learn: 18.7852429 total: 6m 34s remaining: 2m 4s 761: learn: 18.7849826 total: 6m 35s remaining: 2m 3s 762: learn: 18.7839241 total: 6m 35s remaining: 2m 2s 763: learn: 18.7811398 total: 6m 36s remaining: 2m 2s 764: learn: 18.7745030 total: 6m 36s remaining: 2m 1s 765: learn: 18.7742707 total: 6m 37s remaining: 2m 1s 766: learn: 18.7695785 total: 6m 37s remaining: 2m 767: learn: 18.7693790 total: 6m 38s remaining: 2m 768: learn: 18.7602255 total: 6m 38s remaining: 1m 59s 769: learn: 18.7566742 total: 6m 39s remaining: 1m 59s 770: learn: 18.7539377 total: 6m 40s remaining: 1m 58s 771: learn: 18.7510714 total: 6m 40s remaining: 1m 58s 772: learn: 18.7453772 total: 6m 41s remaining: 1m 57s 773: learn: 18.7452140 total: 6m 41s remaining: 1m 57s 774: learn: 18.7440517 total: 6m 42s remaining: 1m 56s 775: learn: 18.7439281 total: 6m 42s remaining: 1m 56s 776: learn: 18.7428828 total: 6m 43s remaining: 1m 55s 777: learn: 18.7420100 total: 6m 43s remaining: 1m 55s 778: learn: 18.7400060 total: 6m 44s remaining: 1m 54s 779: learn: 18.7395276 total: 6m 44s remaining: 1m 54s 780: learn: 18.7318078 total: 6m 45s remaining: 1m 53s 781: learn: 18.7297215 total: 6m 45s remaining: 1m 53s 782: learn: 18.7252043 total: 6m 45s remaining: 1m 52s 783: learn: 18.7234739 total: 6m 46s remaining: 1m 51s 784: learn: 18.7201283 total: 6m 47s remaining: 1m 51s 785: learn: 18.7199616 total: 6m 47s remaining: 1m 50s 786: learn: 18.7174969 total: 6m 48s remaining: 1m 50s 787: learn: 18.7012741 total: 6m 48s remaining: 1m 49s 788: learn: 18.7011520 total: 6m 49s remaining: 1m 49s 789: learn: 18.6956984 total: 6m 49s remaining: 1m 48s 790: learn: 18.6944874 total: 6m 49s remaining: 1m 48s 791: learn: 18.6913593 total: 6m 50s remaining: 1m 47s 792: learn: 18.6906302 total: 6m 51s remaining: 1m 47s 793: learn: 18.6873470 total: 6m 51s remaining: 1m 46s 794: learn: 18.6822229 total: 6m 52s remaining: 1m 46s 795: learn: 18.6772807 total: 6m 52s remaining: 1m 45s 796: learn: 18.6734397 total: 6m 53s remaining: 1m 45s 797: learn: 18.6712212 total: 6m 53s remaining: 1m 44s 798: learn: 18.6693870 total: 6m 54s remaining: 1m 44s 799: learn: 18.6667505 total: 6m 54s remaining: 1m 43s 800: learn: 18.6649930 total: 6m 55s remaining: 1m 43s 801: learn: 18.6647851 total: 6m 55s remaining: 1m 42s 802: learn: 18.6646328 total: 6m 56s remaining: 1m 42s 803: learn: 18.6639955 total: 6m 56s remaining: 1m 41s 804: learn: 18.6582268 total: 6m 57s remaining: 1m 41s 805: learn: 18.6564505 total: 6m 57s remaining: 1m 40s 806: learn: 18.6548617 total: 6m 58s remaining: 1m 40s 807: learn: 18.6528438 total: 6m 58s remaining: 1m 39s 808: learn: 18.6502871 total: 6m 59s remaining: 1m 39s 809: learn: 18.6499583 total: 6m 59s remaining: 1m 38s 810: learn: 18.6495406 total: 7m remaining: 1m 37s 811: learn: 18.6468399 total: 7m 1s remaining: 1m 37s 812: learn: 18.6450477 total: 7m 1s remaining: 1m 36s 813: learn: 18.6389575 total: 7m 2s remaining: 1m 36s 814: learn: 18.6388482 total: 7m 2s remaining: 1m 35s 815: learn: 18.6352942 total: 7m 3s remaining: 1m 35s 816: learn: 18.6258884 total: 7m 3s remaining: 1m 34s 817: learn: 18.6242605 total: 7m 4s remaining: 1m 34s 818: learn: 18.6238195 total: 7m 5s remaining: 1m 33s 819: learn: 18.6211964 total: 7m 5s remaining: 1m 33s 820: learn: 18.6201577 total: 7m 6s remaining: 1m 32s 821: learn: 18.6033667 total: 7m 6s remaining: 1m 32s 822: learn: 18.6032743 total: 7m 7s remaining: 1m 31s 823: learn: 18.5964505 total: 7m 8s remaining: 1m 31s 824: learn: 18.5876723 total: 7m 8s remaining: 1m 30s 825: learn: 18.5816455 total: 7m 9s remaining: 1m 30s 826: learn: 18.5777634 total: 7m 9s remaining: 1m 29s 827: learn: 18.5749083 total: 7m 10s remaining: 1m 29s 828: learn: 18.5131642 total: 7m 10s remaining: 1m 28s 829: learn: 18.5124369 total: 7m 11s remaining: 1m 28s 830: learn: 18.5116074 total: 7m 12s remaining: 1m 27s 831: learn: 18.5032735 total: 7m 12s remaining: 1m 27s 832: learn: 18.4973538 total: 7m 13s remaining: 1m 26s 833: learn: 18.4832434 total: 7m 14s remaining: 1m 26s 834: learn: 18.4811225 total: 7m 14s remaining: 1m 25s 835: learn: 18.4761798 total: 7m 15s remaining: 1m 25s 836: learn: 18.4726901 total: 7m 15s remaining: 1m 24s 837: learn: 18.4329621 total: 7m 16s remaining: 1m 24s 838: learn: 18.4323591 total: 7m 16s remaining: 1m 23s 839: learn: 18.4317741 total: 7m 17s remaining: 1m 23s 840: learn: 18.4307654 total: 7m 17s remaining: 1m 22s 841: learn: 18.4216351 total: 7m 18s remaining: 1m 22s 842: learn: 18.4204295 total: 7m 18s remaining: 1m 21s 843: learn: 18.4199960 total: 7m 19s remaining: 1m 21s 844: learn: 18.4171100 total: 7m 19s remaining: 1m 20s 845: learn: 18.4125365 total: 7m 20s remaining: 1m 20s 846: learn: 18.4121010 total: 7m 20s remaining: 1m 19s 847: learn: 18.4052701 total: 7m 21s remaining: 1m 19s 848: learn: 18.4043881 total: 7m 21s remaining: 1m 18s 849: learn: 18.4041568 total: 7m 22s remaining: 1m 18s 850: learn: 18.4038505 total: 7m 22s remaining: 1m 17s 851: learn: 18.4036706 total: 7m 23s remaining: 1m 16s 852: learn: 18.4032893 total: 7m 23s remaining: 1m 16s 853: learn: 18.3943171 total: 7m 24s remaining: 1m 15s 854: learn: 18.3901397 total: 7m 24s remaining: 1m 15s 855: learn: 18.3896245 total: 7m 24s remaining: 1m 14s 856: learn: 18.3894097 total: 7m 25s remaining: 1m 14s 857: learn: 18.3861096 total: 7m 25s remaining: 1m 13s 858: learn: 18.3783472 total: 7m 26s remaining: 1m 13s 859: learn: 18.3719697 total: 7m 26s remaining: 1m 12s 860: learn: 18.3709561 total: 7m 27s remaining: 1m 12s 861: learn: 18.3639857 total: 7m 27s remaining: 1m 11s 862: learn: 18.3558859 total: 7m 28s remaining: 1m 11s 863: learn: 18.3490277 total: 7m 28s remaining: 1m 10s 864: learn: 18.3486624 total: 7m 29s remaining: 1m 10s 865: learn: 18.3445151 total: 7m 29s remaining: 1m 9s 866: learn: 18.3369658 total: 7m 30s remaining: 1m 9s 867: learn: 18.3317076 total: 7m 30s remaining: 1m 8s 868: learn: 18.3295434 total: 7m 31s remaining: 1m 8s 869: learn: 18.3229790 total: 7m 31s remaining: 1m 7s 870: learn: 18.3195091 total: 7m 32s remaining: 1m 6s 871: learn: 18.3187325 total: 7m 32s remaining: 1m 6s 872: learn: 18.3176776 total: 7m 32s remaining: 1m 5s 873: learn: 18.3169559 total: 7m 33s remaining: 1m 5s 874: learn: 18.3164171 total: 7m 33s remaining: 1m 4s 875: learn: 18.3114749 total: 7m 34s remaining: 1m 4s 876: learn: 18.3113151 total: 7m 34s remaining: 1m 3s 877: learn: 18.3088598 total: 7m 35s remaining: 1m 3s 878: learn: 18.3085266 total: 7m 35s remaining: 1m 2s 879: learn: 18.3084196 total: 7m 36s remaining: 1m 2s 880: learn: 18.3069119 total: 7m 36s remaining: 1m 1s 881: learn: 18.3006242 total: 7m 37s remaining: 1m 1s 882: learn: 18.2969810 total: 7m 37s remaining: 1m 883: learn: 18.2946307 total: 7m 38s remaining: 1m 884: learn: 18.2941245 total: 7m 38s remaining: 59.6s 885: learn: 18.2932217 total: 7m 39s remaining: 59.1s 886: learn: 18.2909055 total: 7m 39s remaining: 58.6s 887: learn: 18.2904823 total: 7m 40s remaining: 58.1s 888: learn: 18.2850314 total: 7m 41s remaining: 57.6s 889: learn: 18.2844913 total: 7m 41s remaining: 57.1s 890: learn: 18.2838156 total: 7m 42s remaining: 56.6s 891: learn: 18.2753787 total: 7m 42s remaining: 56s 892: learn: 18.2736478 total: 7m 43s remaining: 55.5s 893: learn: 18.2690983 total: 7m 43s remaining: 55s 894: learn: 18.2642296 total: 7m 44s remaining: 54.5s 895: learn: 18.2460150 total: 7m 44s remaining: 54s 896: learn: 18.2454463 total: 7m 45s remaining: 53.5s 897: learn: 18.2452435 total: 7m 46s remaining: 52.9s 898: learn: 18.2451758 total: 7m 46s remaining: 52.4s 899: learn: 18.2444805 total: 7m 47s remaining: 51.9s 900: learn: 18.2435120 total: 7m 47s remaining: 51.4s 901: learn: 18.2387194 total: 7m 48s remaining: 50.9s 902: learn: 18.2364902 total: 7m 49s remaining: 50.4s 903: learn: 18.2314178 total: 7m 49s remaining: 49.9s 904: learn: 18.2259325 total: 7m 50s remaining: 49.3s 905: learn: 18.2231533 total: 7m 50s remaining: 48.8s 906: learn: 18.2057076 total: 7m 51s remaining: 48.3s 907: learn: 18.2020265 total: 7m 51s remaining: 47.8s 908: learn: 18.1972180 total: 7m 52s remaining: 47.3s 909: learn: 18.1916547 total: 7m 52s remaining: 46.8s 910: learn: 18.1877930 total: 7m 53s remaining: 46.3s 911: learn: 18.1826160 total: 7m 54s remaining: 45.8s 912: learn: 18.1730166 total: 7m 54s remaining: 45.3s 913: learn: 18.1712228 total: 7m 55s remaining: 44.7s 914: learn: 18.1678491 total: 7m 56s remaining: 44.2s 915: learn: 18.1645069 total: 7m 56s remaining: 43.7s 916: learn: 18.1616436 total: 7m 57s remaining: 43.2s 917: learn: 18.1614120 total: 7m 58s remaining: 42.7s 918: learn: 18.1609070 total: 7m 58s remaining: 42.2s 919: learn: 18.1600031 total: 7m 59s remaining: 41.7s 920: learn: 18.1578617 total: 7m 59s remaining: 41.2s 921: learn: 18.1577867 total: 8m remaining: 40.6s 922: learn: 18.1570922 total: 8m remaining: 40.1s 923: learn: 18.1533801 total: 8m 1s remaining: 39.6s 924: learn: 18.1497496 total: 8m 2s remaining: 39.1s 925: learn: 18.1492591 total: 8m 2s remaining: 38.6s 926: learn: 18.1464120 total: 8m 3s remaining: 38.1s 927: learn: 18.1445224 total: 8m 4s remaining: 37.6s 928: learn: 18.1328925 total: 8m 4s remaining: 37.1s 929: learn: 18.1327029 total: 8m 5s remaining: 36.5s 930: learn: 18.1318236 total: 8m 6s remaining: 36s 931: learn: 18.1314737 total: 8m 6s remaining: 35.5s 932: learn: 18.1280599 total: 8m 7s remaining: 35s 933: learn: 18.1273480 total: 8m 8s remaining: 34.5s 934: learn: 18.1260331 total: 8m 8s remaining: 34s 935: learn: 18.1245838 total: 8m 9s remaining: 33.4s 936: learn: 18.1230432 total: 8m 9s remaining: 32.9s 937: learn: 18.1218440 total: 8m 10s remaining: 32.4s 938: learn: 18.1197346 total: 8m 10s remaining: 31.9s 939: learn: 18.1184925 total: 8m 11s remaining: 31.4s 940: learn: 18.1170982 total: 8m 11s remaining: 30.8s 941: learn: 18.1159417 total: 8m 12s remaining: 30.3s 942: learn: 18.1136299 total: 8m 13s remaining: 29.8s 943: learn: 18.1031597 total: 8m 13s remaining: 29.3s 944: learn: 18.0956072 total: 8m 14s remaining: 28.8s 945: learn: 18.0936245 total: 8m 14s remaining: 28.2s 946: learn: 18.0857056 total: 8m 15s remaining: 27.7s 947: learn: 18.0848276 total: 8m 15s remaining: 27.2s 948: learn: 18.0800671 total: 8m 16s remaining: 26.7s 949: learn: 18.0794823 total: 8m 16s remaining: 26.1s 950: learn: 18.0777102 total: 8m 17s remaining: 25.6s 951: learn: 18.0743413 total: 8m 17s remaining: 25.1s 952: learn: 18.0732552 total: 8m 18s remaining: 24.6s 953: learn: 18.0731623 total: 8m 19s remaining: 24.1s 954: learn: 18.0647836 total: 8m 19s remaining: 23.5s 955: learn: 18.0647133 total: 8m 20s remaining: 23s 956: learn: 18.0596895 total: 8m 20s remaining: 22.5s 957: learn: 18.0579254 total: 8m 21s remaining: 22s 958: learn: 18.0566241 total: 8m 21s remaining: 21.5s 959: learn: 18.0538525 total: 8m 22s remaining: 20.9s 960: learn: 18.0511338 total: 8m 23s remaining: 20.4s 961: learn: 18.0509041 total: 8m 23s remaining: 19.9s 962: learn: 18.0472693 total: 8m 24s remaining: 19.4s 963: learn: 18.0434299 total: 8m 25s remaining: 18.9s 964: learn: 18.0425184 total: 8m 25s remaining: 18.3s 965: learn: 18.0373006 total: 8m 26s remaining: 17.8s 966: learn: 18.0326654 total: 8m 26s remaining: 17.3s 967: learn: 18.0321665 total: 8m 27s remaining: 16.8s 968: learn: 18.0289171 total: 8m 27s remaining: 16.2s 969: learn: 18.0281932 total: 8m 28s remaining: 15.7s 970: learn: 18.0276996 total: 8m 28s remaining: 15.2s 971: learn: 18.0252675 total: 8m 29s remaining: 14.7s 972: learn: 18.0209043 total: 8m 29s remaining: 14.1s 973: learn: 18.0206907 total: 8m 30s remaining: 13.6s 974: learn: 18.0151468 total: 8m 30s remaining: 13.1s 975: learn: 18.0140174 total: 8m 31s remaining: 12.6s 976: learn: 18.0092374 total: 8m 32s remaining: 12.1s 977: learn: 18.0007212 total: 8m 32s remaining: 11.5s 978: learn: 17.9983294 total: 8m 33s remaining: 11s 979: learn: 17.9963143 total: 8m 33s remaining: 10.5s 980: learn: 17.9923584 total: 8m 34s remaining: 9.96s 981: learn: 17.9907492 total: 8m 34s remaining: 9.43s 982: learn: 17.9904658 total: 8m 35s remaining: 8.91s 983: learn: 17.9895575 total: 8m 35s remaining: 8.38s 984: learn: 17.9864493 total: 8m 36s remaining: 7.86s 985: learn: 17.9829849 total: 8m 36s remaining: 7.34s 986: learn: 17.9826168 total: 8m 37s remaining: 6.81s 987: learn: 17.9792986 total: 8m 37s remaining: 6.29s 988: learn: 17.9677505 total: 8m 38s remaining: 5.76s 989: learn: 17.9635089 total: 8m 38s remaining: 5.24s 990: learn: 17.9622231 total: 8m 39s remaining: 4.72s 991: learn: 17.9618284 total: 8m 39s remaining: 4.19s 992: learn: 17.9616067 total: 8m 40s remaining: 3.67s 993: learn: 17.9562983 total: 8m 40s remaining: 3.14s 994: learn: 17.9562802 total: 8m 41s remaining: 2.62s 995: learn: 17.9546308 total: 8m 41s remaining: 2.1s 996: learn: 17.9514809 total: 8m 42s remaining: 1.57s 997: learn: 17.9512569 total: 8m 42s remaining: 1.05s 998: learn: 17.9500801 total: 8m 43s remaining: 524ms 999: learn: 17.9477990 total: 8m 44s remaining: 0us
<catboost.core.CatBoostRegressor at 0x26a0608daf0>
#Получение ответов
y_pred_solution = model.predict(pool_test_solution)
#Вот так они выглядят
y_pred_solution.astype(int)
array([2, 2, 2, ..., 6, 5, 5])
#Формируем sample_solution для отправки на платформу
test['PATIENT_ID_COUNT'] = y_pred_solution.astype(int)
#Сохраняем в csv файл
test.to_csv('sample_solution.csv', sep=';', index=None)